Note: This is test shard 8 of 8.
[==========] Running 7 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 1 test from TableRangeRebalancingTest
[ RUN ] TableRangeRebalancingTest.Basic
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240501 07:59:05.303288 398 test_util.cc:268] Using random seed: -599259931
I20240501 07:59:05.309824 398 ts_itest-base.cc:112] Starting cluster with:
I20240501 07:59:05.309922 398 ts_itest-base.cc:113] --------------
I20240501 07:59:05.309954 398 ts_itest-base.cc:114] 3 tablet servers
I20240501 07:59:05.309973 398 ts_itest-base.cc:115] 3 replicas per TS
I20240501 07:59:05.309986 398 ts_itest-base.cc:116] --------------
2024-05-01T07:59:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T07:59:05Z Disabled control of system clock
I20240501 07:59:05.318647 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:44485
--webserver_interface=127.0.99.190
--webserver_port=0
--builtin_ntp_servers=127.0.99.148:38385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.99.190:44485
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240501 07:59:05.419019 410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:05.419224 410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:05.419278 410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:05.422179 410 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:05.422243 410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 07:59:05.422274 410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:05.422298 410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 07:59:05.422329 410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 07:59:05.425904 410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:38385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.0.99.190:44485
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:44485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.99.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:05.426721 410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:05.427803 410 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:05.436043 418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:05.436084 415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:05.436553 416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:05.437047 410 server_base.cc:1034] running on GCE node
I20240501 07:59:05.437462 410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:05.438267 410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:05.439414 410 hybrid_clock.cc:648] HybridClock initialized: now 1714550345439375 us; error 51 us; skew 500 ppm
I20240501 07:59:05.447139 410 webserver.cc:460] Webserver started at http://127.0.99.190:41927/ using document root <none> and password file <none>
I20240501 07:59:05.447644 410 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:05.447696 410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:05.447873 410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:05.449431 410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "e85a8f6d576e44b0a1e1381c62e3d14c"
format_stamp: "Formatted at 2024-05-01 07:59:05 on dist-test-slave-f81v"
I20240501 07:59:05.449875 410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "e85a8f6d576e44b0a1e1381c62e3d14c"
format_stamp: "Formatted at 2024-05-01 07:59:05 on dist-test-slave-f81v"
I20240501 07:59:05.452909 410 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240501 07:59:05.454780 424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:05.455657 410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:05.455765 410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
uuid: "e85a8f6d576e44b0a1e1381c62e3d14c"
format_stamp: "Formatted at 2024-05-01 07:59:05 on dist-test-slave-f81v"
I20240501 07:59:05.455838 410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:05.470562 410 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:05.471108 410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:05.471253 410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:05.477319 410 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:44485
I20240501 07:59:05.477322 476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:44485 every 8 connection(s)
I20240501 07:59:05.478183 410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 07:59:05.480532 477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:05.483603 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 410
I20240501 07:59:05.483731 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 07:59:05.485148 477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c: Bootstrap starting.
I20240501 07:59:05.487532 477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:05.488204 477 log.cc:824] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:05.489707 477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c: No bootstrap required, opened a new log
I20240501 07:59:05.492655 477 raft_consensus.cc:348] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85a8f6d576e44b0a1e1381c62e3d14c" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 44485 } }
I20240501 07:59:05.492856 477 raft_consensus.cc:374] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:05.492903 477 raft_consensus.cc:729] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e85a8f6d576e44b0a1e1381c62e3d14c, State: Initialized, Role: FOLLOWER
I20240501 07:59:05.493278 477 consensus_queue.cc:260] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85a8f6d576e44b0a1e1381c62e3d14c" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 44485 } }
I20240501 07:59:05.493392 477 raft_consensus.cc:388] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 07:59:05.493505 477 raft_consensus.cc:482] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 07:59:05.493620 477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:05.494300 477 raft_consensus.cc:504] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85a8f6d576e44b0a1e1381c62e3d14c" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 44485 } }
I20240501 07:59:05.494583 477 leader_election.cc:302] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e85a8f6d576e44b0a1e1381c62e3d14c; no voters:
I20240501 07:59:05.494819 477 leader_election.cc:288] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 07:59:05.495162 482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:05.495355 482 raft_consensus.cc:686] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [term 1 LEADER]: Becoming Leader. State: Replica: e85a8f6d576e44b0a1e1381c62e3d14c, State: Running, Role: LEADER
I20240501 07:59:05.495645 482 consensus_queue.cc:237] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85a8f6d576e44b0a1e1381c62e3d14c" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 44485 } }
I20240501 07:59:05.495893 477 sys_catalog.cc:564] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [sys.catalog]: configured and running, proceeding with master startup.
I20240501 07:59:05.496819 484 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e85a8f6d576e44b0a1e1381c62e3d14c. Latest consensus state: current_term: 1 leader_uuid: "e85a8f6d576e44b0a1e1381c62e3d14c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85a8f6d576e44b0a1e1381c62e3d14c" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 44485 } } }
I20240501 07:59:05.496937 484 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [sys.catalog]: This master's current role is: LEADER
I20240501 07:59:05.497093 483 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e85a8f6d576e44b0a1e1381c62e3d14c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e85a8f6d576e44b0a1e1381c62e3d14c" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 44485 } } }
I20240501 07:59:05.497176 483 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c [sys.catalog]: This master's current role is: LEADER
I20240501 07:59:05.497277 491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 07:59:05.500097 491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 07:59:05.504918 491 catalog_manager.cc:1347] Generated new cluster ID: b5d76c5387df4ec3961389bbff41b7de
I20240501 07:59:05.504976 491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 07:59:05.512549 491 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 07:59:05.513306 491 catalog_manager.cc:1504] Loading token signing keys...
I20240501 07:59:05.525156 491 catalog_manager.cc:5857] T 00000000000000000000000000000000 P e85a8f6d576e44b0a1e1381c62e3d14c: Generated new TSK 0
I20240501 07:59:05.525921 491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 07:59:05.529074 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.129:0
--local_ip_for_outbound_sockets=127.0.99.129
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44485
--builtin_ntp_servers=127.0.99.148:38385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 07:59:05.636610 501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:05.636792 501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:05.636847 501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:05.639756 501 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:05.639822 501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:05.639906 501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.129
I20240501 07:59:05.643348 501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:38385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44485
--never_fsync=true
--heap_profile_path=/tmp/kudu.501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.129
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:05.644173 501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:05.645210 501 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:05.653313 507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:05.653669 506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:05.653796 509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:05.654341 501 server_base.cc:1034] running on GCE node
I20240501 07:59:05.654659 501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:05.655170 501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:05.656324 501 hybrid_clock.cc:648] HybridClock initialized: now 1714550345656313 us; error 28 us; skew 500 ppm
I20240501 07:59:05.658138 501 webserver.cc:460] Webserver started at http://127.0.99.129:39617/ using document root <none> and password file <none>
I20240501 07:59:05.658658 501 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:05.658710 501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:05.658871 501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:05.660522 501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "1e63aa3297ad4acd8856e6aa28008957"
format_stamp: "Formatted at 2024-05-01 07:59:05 on dist-test-slave-f81v"
I20240501 07:59:05.660971 501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "1e63aa3297ad4acd8856e6aa28008957"
format_stamp: "Formatted at 2024-05-01 07:59:05 on dist-test-slave-f81v"
I20240501 07:59:05.664043 501 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 07:59:05.666018 515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:05.666996 501 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:05.667081 501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "1e63aa3297ad4acd8856e6aa28008957"
format_stamp: "Formatted at 2024-05-01 07:59:05 on dist-test-slave-f81v"
I20240501 07:59:05.667151 501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:05.678491 501 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:05.679018 501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:05.679137 501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:05.679566 501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:05.680418 501 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:05.680461 501 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:05.680501 501 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:05.680521 501 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:05.688235 501 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:36459
I20240501 07:59:05.688267 628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:36459 every 8 connection(s)
I20240501 07:59:05.689060 501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 07:59:05.693984 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 501
I20240501 07:59:05.694072 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 07:59:05.696236 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:0
--local_ip_for_outbound_sockets=127.0.99.130
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44485
--builtin_ntp_servers=127.0.99.148:38385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 07:59:05.697463 629 heartbeater.cc:347] Connected to a master server at 127.0.99.190:44485
I20240501 07:59:05.697659 629 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:05.698174 629 heartbeater.cc:511] Master 127.0.99.190:44485 requested a full tablet report, sending...
I20240501 07:59:05.699424 441 ts_manager.cc:194] Registered new tserver with Master: 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459)
I20240501 07:59:05.700554 441 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.129:44827
W20240501 07:59:05.797976 632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:05.798192 632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:05.798245 632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:05.801230 632 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:05.801292 632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:05.801373 632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240501 07:59:05.804797 632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:38385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44485
--never_fsync=true
--heap_profile_path=/tmp/kudu.632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:05.805589 632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:05.807096 632 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:05.814291 640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:05.814839 637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:05.815325 638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:05.815814 632 server_base.cc:1034] running on GCE node
I20240501 07:59:05.816148 632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:05.816637 632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:05.817780 632 hybrid_clock.cc:648] HybridClock initialized: now 1714550345817764 us; error 30 us; skew 500 ppm
I20240501 07:59:05.819442 632 webserver.cc:460] Webserver started at http://127.0.99.130:39343/ using document root <none> and password file <none>
I20240501 07:59:05.819960 632 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:05.820015 632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:05.820192 632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:05.821723 632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "6c6e5fc1b40e4015a97c69fef00a4cff"
format_stamp: "Formatted at 2024-05-01 07:59:05 on dist-test-slave-f81v"
I20240501 07:59:05.822141 632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "6c6e5fc1b40e4015a97c69fef00a4cff"
format_stamp: "Formatted at 2024-05-01 07:59:05 on dist-test-slave-f81v"
I20240501 07:59:05.825078 632 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 07:59:05.826972 646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:05.827829 632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:05.827947 632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "6c6e5fc1b40e4015a97c69fef00a4cff"
format_stamp: "Formatted at 2024-05-01 07:59:05 on dist-test-slave-f81v"
I20240501 07:59:05.828023 632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:05.841568 632 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:05.842093 632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:05.842221 632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:05.842669 632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:05.843452 632 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:05.843497 632 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:05.843544 632 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:05.843569 632 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:05.852499 632 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:43793
I20240501 07:59:05.852538 759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:43793 every 8 connection(s)
I20240501 07:59:05.853335 632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 07:59:05.860841 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 632
I20240501 07:59:05.860900 760 heartbeater.cc:347] Connected to a master server at 127.0.99.190:44485
I20240501 07:59:05.860941 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 07:59:05.861099 760 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:05.861575 760 heartbeater.cc:511] Master 127.0.99.190:44485 requested a full tablet report, sending...
I20240501 07:59:05.862421 441 ts_manager.cc:194] Registered new tserver with Master: 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793)
I20240501 07:59:05.862736 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:0
--local_ip_for_outbound_sockets=127.0.99.131
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44485
--builtin_ntp_servers=127.0.99.148:38385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 07:59:05.862998 441 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:42511
W20240501 07:59:05.963093 763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:05.963264 763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:05.963303 763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:05.966156 763 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:05.966209 763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:05.966270 763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240501 07:59:05.969730 763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:38385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44485
--never_fsync=true
--heap_profile_path=/tmp/kudu.763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:05.970525 763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:05.971560 763 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:05.978662 769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:05.979224 768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:05.979727 771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:05.980222 763 server_base.cc:1034] running on GCE node
I20240501 07:59:05.980535 763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:05.981016 763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:05.982151 763 hybrid_clock.cc:648] HybridClock initialized: now 1714550345982132 us; error 33 us; skew 500 ppm
I20240501 07:59:05.984083 763 webserver.cc:460] Webserver started at http://127.0.99.131:40557/ using document root <none> and password file <none>
I20240501 07:59:05.984586 763 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:05.984637 763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:05.984788 763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:05.986337 763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "8a1a8654e29140f184ff9da4d55aceeb"
format_stamp: "Formatted at 2024-05-01 07:59:05 on dist-test-slave-f81v"
I20240501 07:59:05.986783 763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "8a1a8654e29140f184ff9da4d55aceeb"
format_stamp: "Formatted at 2024-05-01 07:59:05 on dist-test-slave-f81v"
I20240501 07:59:05.990036 763 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20240501 07:59:05.992000 777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:05.992959 763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:05.993055 763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "8a1a8654e29140f184ff9da4d55aceeb"
format_stamp: "Formatted at 2024-05-01 07:59:05 on dist-test-slave-f81v"
I20240501 07:59:05.993135 763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:06.016083 763 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:06.016678 763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:06.016816 763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:06.017321 763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:06.018158 763 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:06.018203 763 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:06.018239 763 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:06.018260 763 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:06.027642 763 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:44687
I20240501 07:59:06.027658 890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:44687 every 8 connection(s)
I20240501 07:59:06.028577 763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 07:59:06.036384 891 heartbeater.cc:347] Connected to a master server at 127.0.99.190:44485
I20240501 07:59:06.036561 891 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:06.036985 891 heartbeater.cc:511] Master 127.0.99.190:44485 requested a full tablet report, sending...
I20240501 07:59:06.037569 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 763
I20240501 07:59:06.037662 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 07:59:06.037858 441 ts_manager.cc:194] Registered new tserver with Master: 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:06.038420 441 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:53877
I20240501 07:59:06.039824 398 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20240501 07:59:06.049067 398 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20240501 07:59:06.702504 629 heartbeater.cc:503] Master 127.0.99.190:44485 was elected leader, sending a full tablet report...
I20240501 07:59:06.864693 760 heartbeater.cc:503] Master 127.0.99.190:44485 was elected leader, sending a full tablet report...
I20240501 07:59:07.040378 891 heartbeater.cc:503] Master 127.0.99.190:44485 was elected leader, sending a full tablet report...
I20240501 07:59:09.801556 441 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35500:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\200\007\001\000\000\000\000\000\006\001\000\000\000\000\000\007\001\000\377\377\377\177"
indirect_data: "<redacted>"""
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 8
seed: 0
}
}
W20240501 07:59:09.804503 441 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240501 07:59:09.824034 551 tablet_service.cc:1467] Processing CreateTablet for tablet 51a1bbf6154345b482f7425a929a01cb (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.823953 553 tablet_service.cc:1467] Processing CreateTablet for tablet f468589b44464314a74d96dd6092603d (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.823596 550 tablet_service.cc:1467] Processing CreateTablet for tablet 606e2456198346be96fcb6b9e71f78ad (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.824913 558 tablet_service.cc:1467] Processing CreateTablet for tablet 07cd57b9e80a4231944fc0d91f363c99 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.825199 557 tablet_service.cc:1467] Processing CreateTablet for tablet 2382ca3cb43a4cff9b4594a9936ac9e9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.825505 560 tablet_service.cc:1467] Processing CreateTablet for tablet 29a7d6078f164c7085e694de0fa3b490 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.827706 685 tablet_service.cc:1467] Processing CreateTablet for tablet befa9b316ce74132882efeee4bb05aa1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.828063 683 tablet_service.cc:1467] Processing CreateTablet for tablet 7dd94d49fbdf4dd695057492c52ec589 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.828387 682 tablet_service.cc:1467] Processing CreateTablet for tablet 51a1bbf6154345b482f7425a929a01cb (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.828024 823 tablet_service.cc:1467] Processing CreateTablet for tablet fe588e82efae459abbe85693d4f77c14 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.827472 684 tablet_service.cc:1467] Processing CreateTablet for tablet f468589b44464314a74d96dd6092603d (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.828645 685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet befa9b316ce74132882efeee4bb05aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.827466 681 tablet_service.cc:1467] Processing CreateTablet for tablet 606e2456198346be96fcb6b9e71f78ad (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.828935 684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f468589b44464314a74d96dd6092603d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.828948 823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe588e82efae459abbe85693d4f77c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.829125 682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51a1bbf6154345b482f7425a929a01cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.829726 683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dd94d49fbdf4dd695057492c52ec589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.829769 686 tablet_service.cc:1467] Processing CreateTablet for tablet 1cdacda459cc4daaad6fcde6f07058d5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.829980 686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cdacda459cc4daaad6fcde6f07058d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.830462 687 tablet_service.cc:1467] Processing CreateTablet for tablet 4b3f550660c441f381045fd89143ec14 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.830682 687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b3f550660c441f381045fd89143ec14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.831398 691 tablet_service.cc:1467] Processing CreateTablet for tablet 29a7d6078f164c7085e694de0fa3b490 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.831632 691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29a7d6078f164c7085e694de0fa3b490. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.832098 681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 606e2456198346be96fcb6b9e71f78ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.832697 559 tablet_service.cc:1467] Processing CreateTablet for tablet ae1c471e7c9d4965b500bbcfcd0e0922 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.832944 688 tablet_service.cc:1467] Processing CreateTablet for tablet 2382ca3cb43a4cff9b4594a9936ac9e9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.832996 561 tablet_service.cc:1467] Processing CreateTablet for tablet fe588e82efae459abbe85693d4f77c14 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.833315 815 tablet_service.cc:1467] Processing CreateTablet for tablet f468589b44464314a74d96dd6092603d (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.833336 689 tablet_service.cc:1467] Processing CreateTablet for tablet 07cd57b9e80a4231944fc0d91f363c99 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.833546 815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f468589b44464314a74d96dd6092603d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.833703 690 tablet_service.cc:1467] Processing CreateTablet for tablet ae1c471e7c9d4965b500bbcfcd0e0922 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.833909 822 tablet_service.cc:1467] Processing CreateTablet for tablet 29a7d6078f164c7085e694de0fa3b490 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.834124 822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29a7d6078f164c7085e694de0fa3b490. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.834097 692 tablet_service.cc:1467] Processing CreateTablet for tablet fe588e82efae459abbe85693d4f77c14 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.824774 551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51a1bbf6154345b482f7425a929a01cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.834833 558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07cd57b9e80a4231944fc0d91f363c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.835352 557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2382ca3cb43a4cff9b4594a9936ac9e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.838261 814 tablet_service.cc:1467] Processing CreateTablet for tablet 7dd94d49fbdf4dd695057492c52ec589 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.838490 814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dd94d49fbdf4dd695057492c52ec589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.839713 911 tablet_bootstrap.cc:492] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.839813 553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f468589b44464314a74d96dd6092603d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.840988 816 tablet_service.cc:1467] Processing CreateTablet for tablet befa9b316ce74132882efeee4bb05aa1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.841240 816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet befa9b316ce74132882efeee4bb05aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.841359 909 tablet_bootstrap.cc:492] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.841885 550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 606e2456198346be96fcb6b9e71f78ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.843010 911 tablet_bootstrap.cc:654] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.842985 818 tablet_service.cc:1467] Processing CreateTablet for tablet 4b3f550660c441f381045fd89143ec14 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.843039 819 tablet_service.cc:1467] Processing CreateTablet for tablet 2382ca3cb43a4cff9b4594a9936ac9e9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.843226 818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b3f550660c441f381045fd89143ec14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.843422 819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2382ca3cb43a4cff9b4594a9936ac9e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.843724 817 tablet_service.cc:1467] Processing CreateTablet for tablet 1cdacda459cc4daaad6fcde6f07058d5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.843961 817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cdacda459cc4daaad6fcde6f07058d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.844635 911 log.cc:824] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:09.845804 813 tablet_service.cc:1467] Processing CreateTablet for tablet 51a1bbf6154345b482f7425a929a01cb (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.846030 813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51a1bbf6154345b482f7425a929a01cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.827468 694 tablet_service.cc:1467] Processing CreateTablet for tablet b69525d893854177832cf2e3f8066170 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.847831 820 tablet_service.cc:1467] Processing CreateTablet for tablet 07cd57b9e80a4231944fc0d91f363c99 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.848078 820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07cd57b9e80a4231944fc0d91f363c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.849067 821 tablet_service.cc:1467] Processing CreateTablet for tablet ae1c471e7c9d4965b500bbcfcd0e0922 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.849308 821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae1c471e7c9d4965b500bbcfcd0e0922. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.849606 911 tablet_bootstrap.cc:492] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.849830 561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe588e82efae459abbe85693d4f77c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.849912 911 ts_tablet_manager.cc:1367] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20240501 07:59:09.824101 555 tablet_service.cc:1467] Processing CreateTablet for tablet 1cdacda459cc4daaad6fcde6f07058d5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.851109 555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cdacda459cc4daaad6fcde6f07058d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.823688 554 tablet_service.cc:1467] Processing CreateTablet for tablet befa9b316ce74132882efeee4bb05aa1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.852209 554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet befa9b316ce74132882efeee4bb05aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.853741 555 tablet_service.cc:1467] Processing CreateTablet for tablet 917d5a979ab44260838db6484408c453 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.853955 555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917d5a979ab44260838db6484408c453. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.854173 560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29a7d6078f164c7085e694de0fa3b490. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.856870 821 tablet_service.cc:1467] Processing CreateTablet for tablet 606e2456198346be96fcb6b9e71f78ad (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.856937 559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae1c471e7c9d4965b500bbcfcd0e0922. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.857098 821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 606e2456198346be96fcb6b9e71f78ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.857753 693 tablet_service.cc:1467] Processing CreateTablet for tablet 2ceab1e690b845fd974f0f7dd563c768 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.857518 911 raft_consensus.cc:348] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.827785 825 tablet_service.cc:1467] Processing CreateTablet for tablet b69525d893854177832cf2e3f8066170 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.857966 911 raft_consensus.cc:374] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.858108 911 raft_consensus.cc:729] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.858186 825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b69525d893854177832cf2e3f8066170. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.858708 818 tablet_service.cc:1467] Processing CreateTablet for tablet 5b2abd6fa45746aebd6eba231483ba4f (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.858917 818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b2abd6fa45746aebd6eba231483ba4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.859033 687 tablet_service.cc:1467] Processing CreateTablet for tablet 5b2abd6fa45746aebd6eba231483ba4f (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.859721 684 tablet_service.cc:1467] Processing CreateTablet for tablet 917d5a979ab44260838db6484408c453 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.860071 687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b2abd6fa45746aebd6eba231483ba4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.860247 689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07cd57b9e80a4231944fc0d91f363c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.860508 559 tablet_service.cc:1467] Processing CreateTablet for tablet 5b2abd6fa45746aebd6eba231483ba4f (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.860728 559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b2abd6fa45746aebd6eba231483ba4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.861052 820 tablet_service.cc:1467] Processing CreateTablet for tablet 917d5a979ab44260838db6484408c453 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.861254 820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917d5a979ab44260838db6484408c453. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.861863 692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe588e82efae459abbe85693d4f77c14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.863477 690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae1c471e7c9d4965b500bbcfcd0e0922. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.863624 684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917d5a979ab44260838db6484408c453. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.864001 688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2382ca3cb43a4cff9b4594a9936ac9e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.860069 911 consensus_queue.cc:260] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.867199 563 tablet_service.cc:1467] Processing CreateTablet for tablet b69525d893854177832cf2e3f8066170 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 07:59:09.867434 563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b69525d893854177832cf2e3f8066170. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.867897 911 ts_tablet_manager.cc:1398] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20240501 07:59:09.868165 911 tablet_bootstrap.cc:492] T f468589b44464314a74d96dd6092603d P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.868891 693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ceab1e690b845fd974f0f7dd563c768. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.823472 556 tablet_service.cc:1467] Processing CreateTablet for tablet 4b3f550660c441f381045fd89143ec14 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.823472 552 tablet_service.cc:1467] Processing CreateTablet for tablet 7dd94d49fbdf4dd695057492c52ec589 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.870574 911 tablet_bootstrap.cc:654] T f468589b44464314a74d96dd6092603d P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.870752 556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b3f550660c441f381045fd89143ec14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.870986 552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dd94d49fbdf4dd695057492c52ec589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.871124 910 tablet_bootstrap.cc:492] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.872154 911 tablet_bootstrap.cc:492] T f468589b44464314a74d96dd6092603d P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.872223 911 ts_tablet_manager.cc:1367] T f468589b44464314a74d96dd6092603d P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 07:59:09.827795 824 tablet_service.cc:1467] Processing CreateTablet for tablet 2ceab1e690b845fd974f0f7dd563c768 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.872725 824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ceab1e690b845fd974f0f7dd563c768. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.872710 911 raft_consensus.cc:348] T f468589b44464314a74d96dd6092603d P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.872802 911 raft_consensus.cc:374] T f468589b44464314a74d96dd6092603d P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.872828 911 raft_consensus.cc:729] T f468589b44464314a74d96dd6092603d P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.872932 911 consensus_queue.cc:260] T f468589b44464314a74d96dd6092603d P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.873157 911 ts_tablet_manager.cc:1398] T f468589b44464314a74d96dd6092603d P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:09.873296 911 tablet_bootstrap.cc:492] T 606e2456198346be96fcb6b9e71f78ad P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.873302 910 tablet_bootstrap.cc:654] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.874049 911 tablet_bootstrap.cc:654] T 606e2456198346be96fcb6b9e71f78ad P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.874083 910 log.cc:824] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:09.874773 694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b69525d893854177832cf2e3f8066170. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.875214 911 tablet_bootstrap.cc:492] T 606e2456198346be96fcb6b9e71f78ad P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.875288 911 ts_tablet_manager.cc:1367] T 606e2456198346be96fcb6b9e71f78ad P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.875654 911 raft_consensus.cc:348] T 606e2456198346be96fcb6b9e71f78ad P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.875744 911 raft_consensus.cc:374] T 606e2456198346be96fcb6b9e71f78ad P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.875772 911 raft_consensus.cc:729] T 606e2456198346be96fcb6b9e71f78ad P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.875872 911 consensus_queue.cc:260] T 606e2456198346be96fcb6b9e71f78ad P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.876147 911 ts_tablet_manager.cc:1398] T 606e2456198346be96fcb6b9e71f78ad P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.876309 911 tablet_bootstrap.cc:492] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.876711 910 tablet_bootstrap.cc:492] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.876884 910 ts_tablet_manager.cc:1367] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20240501 07:59:09.823472 562 tablet_service.cc:1467] Processing CreateTablet for tablet 2ceab1e690b845fd974f0f7dd563c768 (DEFAULT_TABLE table=rebalance_test_table_0 [id=631b64057b714675a848a47a27a70075]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 07:59:09.877514 562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ceab1e690b845fd974f0f7dd563c768. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:09.877552 911 tablet_bootstrap.cc:654] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.878508 911 tablet_bootstrap.cc:492] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.878588 911 ts_tablet_manager.cc:1367] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:09.878957 911 raft_consensus.cc:348] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.879060 911 raft_consensus.cc:374] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.879096 911 raft_consensus.cc:729] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.879230 911 consensus_queue.cc:260] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.879526 911 ts_tablet_manager.cc:1398] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.879675 911 tablet_bootstrap.cc:492] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.880471 911 tablet_bootstrap.cc:654] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.881028 910 raft_consensus.cc:348] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.881286 910 raft_consensus.cc:374] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.881376 910 raft_consensus.cc:729] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.881408 911 tablet_bootstrap.cc:492] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.881469 911 ts_tablet_manager.cc:1367] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.881464 909 tablet_bootstrap.cc:654] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.881855 911 raft_consensus.cc:348] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.881950 911 raft_consensus.cc:374] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.881978 911 raft_consensus.cc:729] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.881953 910 consensus_queue.cc:260] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.882100 911 consensus_queue.cc:260] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.882128 909 log.cc:824] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:09.882330 911 ts_tablet_manager.cc:1398] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.882477 911 tablet_bootstrap.cc:492] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.882862 910 ts_tablet_manager.cc:1398] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240501 07:59:09.883255 911 tablet_bootstrap.cc:654] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.883807 909 tablet_bootstrap.cc:492] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.883980 909 ts_tablet_manager.cc:1367] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.004s
I20240501 07:59:09.885353 910 tablet_bootstrap.cc:492] T 7dd94d49fbdf4dd695057492c52ec589 P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.885358 911 tablet_bootstrap.cc:492] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.885417 911 ts_tablet_manager.cc:1367] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:09.885783 911 raft_consensus.cc:348] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.885869 911 raft_consensus.cc:374] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.885895 911 raft_consensus.cc:729] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.885990 911 consensus_queue.cc:260] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.886183 910 tablet_bootstrap.cc:654] T 7dd94d49fbdf4dd695057492c52ec589 P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.886214 911 ts_tablet_manager.cc:1398] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.886363 911 tablet_bootstrap.cc:492] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.887110 911 tablet_bootstrap.cc:654] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.887454 910 tablet_bootstrap.cc:492] T 7dd94d49fbdf4dd695057492c52ec589 P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.887521 910 ts_tablet_manager.cc:1367] T 7dd94d49fbdf4dd695057492c52ec589 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:09.887625 909 raft_consensus.cc:348] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.887813 909 raft_consensus.cc:374] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.887862 909 raft_consensus.cc:729] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.887989 910 raft_consensus.cc:348] T 7dd94d49fbdf4dd695057492c52ec589 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.888077 910 raft_consensus.cc:374] T 7dd94d49fbdf4dd695057492c52ec589 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.888106 910 raft_consensus.cc:729] T 7dd94d49fbdf4dd695057492c52ec589 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.888212 910 consensus_queue.cc:260] T 7dd94d49fbdf4dd695057492c52ec589 P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.888442 910 ts_tablet_manager.cc:1398] T 7dd94d49fbdf4dd695057492c52ec589 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.888475 911 tablet_bootstrap.cc:492] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.888417 909 consensus_queue.cc:260] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.888545 911 ts_tablet_manager.cc:1367] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.888595 910 tablet_bootstrap.cc:492] T befa9b316ce74132882efeee4bb05aa1 P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.888885 911 raft_consensus.cc:348] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.888981 911 raft_consensus.cc:374] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.889012 911 raft_consensus.cc:729] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.889123 911 consensus_queue.cc:260] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.889237 909 ts_tablet_manager.cc:1398] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240501 07:59:09.889375 910 tablet_bootstrap.cc:654] T befa9b316ce74132882efeee4bb05aa1 P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.889417 911 ts_tablet_manager.cc:1398] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.889539 909 tablet_bootstrap.cc:492] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.889559 911 tablet_bootstrap.cc:492] T befa9b316ce74132882efeee4bb05aa1 P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.890179 913 raft_consensus.cc:482] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:09.890308 913 raft_consensus.cc:504] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.890388 909 tablet_bootstrap.cc:654] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.890429 911 tablet_bootstrap.cc:654] T befa9b316ce74132882efeee4bb05aa1 P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.891397 913 leader_election.cc:288] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:09.891592 909 tablet_bootstrap.cc:492] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.891659 909 ts_tablet_manager.cc:1367] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.892163 909 raft_consensus.cc:348] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.892256 909 raft_consensus.cc:374] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.892284 909 raft_consensus.cc:729] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.892395 909 consensus_queue.cc:260] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.892617 909 ts_tablet_manager.cc:1398] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.892771 909 tablet_bootstrap.cc:492] T 1cdacda459cc4daaad6fcde6f07058d5 P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.893613 909 tablet_bootstrap.cc:654] T 1cdacda459cc4daaad6fcde6f07058d5 P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.894996 910 tablet_bootstrap.cc:492] T befa9b316ce74132882efeee4bb05aa1 P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.895073 910 ts_tablet_manager.cc:1367] T befa9b316ce74132882efeee4bb05aa1 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240501 07:59:09.895520 909 tablet_bootstrap.cc:492] T 1cdacda459cc4daaad6fcde6f07058d5 P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.895470 910 raft_consensus.cc:348] T befa9b316ce74132882efeee4bb05aa1 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.895573 910 raft_consensus.cc:374] T befa9b316ce74132882efeee4bb05aa1 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.895591 909 ts_tablet_manager.cc:1367] T 1cdacda459cc4daaad6fcde6f07058d5 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:09.895603 910 raft_consensus.cc:729] T befa9b316ce74132882efeee4bb05aa1 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.895720 910 consensus_queue.cc:260] T befa9b316ce74132882efeee4bb05aa1 P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.896005 909 raft_consensus.cc:348] T 1cdacda459cc4daaad6fcde6f07058d5 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.896039 910 ts_tablet_manager.cc:1398] T befa9b316ce74132882efeee4bb05aa1 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.896090 909 raft_consensus.cc:374] T 1cdacda459cc4daaad6fcde6f07058d5 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.896117 909 raft_consensus.cc:729] T 1cdacda459cc4daaad6fcde6f07058d5 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.896241 909 consensus_queue.cc:260] T 1cdacda459cc4daaad6fcde6f07058d5 P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.896492 909 ts_tablet_manager.cc:1398] T 1cdacda459cc4daaad6fcde6f07058d5 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.897419 910 tablet_bootstrap.cc:492] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.898048 909 tablet_bootstrap.cc:492] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.898278 910 tablet_bootstrap.cc:654] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.898578 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07cd57b9e80a4231944fc0d91f363c99" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" is_pre_election: true
I20240501 07:59:09.898851 909 tablet_bootstrap.cc:654] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
W20240501 07:59:09.899257 519 leader_election.cc:341] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793): Illegal state: must be running to vote when last-logged opid is not known
I20240501 07:59:09.899600 845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07cd57b9e80a4231944fc0d91f363c99" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:09.899763 911 tablet_bootstrap.cc:492] T befa9b316ce74132882efeee4bb05aa1 P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.899834 911 ts_tablet_manager.cc:1367] T befa9b316ce74132882efeee4bb05aa1 P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240501 07:59:09.899919 909 tablet_bootstrap.cc:492] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.900020 909 ts_tablet_manager.cc:1367] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:09.900377 909 raft_consensus.cc:348] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.900377 911 raft_consensus.cc:348] T befa9b316ce74132882efeee4bb05aa1 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.900467 911 raft_consensus.cc:374] T befa9b316ce74132882efeee4bb05aa1 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.900511 911 raft_consensus.cc:729] T befa9b316ce74132882efeee4bb05aa1 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.900521 909 raft_consensus.cc:374] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.900591 909 raft_consensus.cc:729] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.900642 911 consensus_queue.cc:260] T befa9b316ce74132882efeee4bb05aa1 P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.900890 911 ts_tablet_manager.cc:1398] T befa9b316ce74132882efeee4bb05aa1 P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.900854 909 consensus_queue.cc:260] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.901072 911 tablet_bootstrap.cc:492] T 29a7d6078f164c7085e694de0fa3b490 P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.901144 909 ts_tablet_manager.cc:1398] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.901271 909 tablet_bootstrap.cc:492] T f468589b44464314a74d96dd6092603d P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
W20240501 07:59:09.901500 519 leader_election.cc:341] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687): Illegal state: must be running to vote when last-logged opid is not known
I20240501 07:59:09.901582 519 leader_election.cc:302] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957; no voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb
I20240501 07:59:09.901803 925 raft_consensus.cc:2726] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 07:59:09.901857 911 tablet_bootstrap.cc:654] T 29a7d6078f164c7085e694de0fa3b490 P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.902287 909 tablet_bootstrap.cc:654] T f468589b44464314a74d96dd6092603d P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.902801 910 tablet_bootstrap.cc:492] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.902881 910 ts_tablet_manager.cc:1367] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 07:59:09.902928 911 tablet_bootstrap.cc:492] T 29a7d6078f164c7085e694de0fa3b490 P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.902995 911 ts_tablet_manager.cc:1367] T 29a7d6078f164c7085e694de0fa3b490 P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.903204 909 tablet_bootstrap.cc:492] T f468589b44464314a74d96dd6092603d P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.903266 909 ts_tablet_manager.cc:1367] T f468589b44464314a74d96dd6092603d P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.903266 910 raft_consensus.cc:348] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.903352 910 raft_consensus.cc:374] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.903381 910 raft_consensus.cc:729] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.903365 911 raft_consensus.cc:348] T 29a7d6078f164c7085e694de0fa3b490 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.903445 911 raft_consensus.cc:374] T 29a7d6078f164c7085e694de0fa3b490 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.903473 911 raft_consensus.cc:729] T 29a7d6078f164c7085e694de0fa3b490 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.903573 911 consensus_queue.cc:260] T 29a7d6078f164c7085e694de0fa3b490 P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.903635 909 raft_consensus.cc:348] T f468589b44464314a74d96dd6092603d P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.903731 909 raft_consensus.cc:374] T f468589b44464314a74d96dd6092603d P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.903769 909 raft_consensus.cc:729] T f468589b44464314a74d96dd6092603d P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.903825 911 ts_tablet_manager.cc:1398] T 29a7d6078f164c7085e694de0fa3b490 P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.903625 910 consensus_queue.cc:260] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.903878 909 consensus_queue.cc:260] T f468589b44464314a74d96dd6092603d P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.903996 911 tablet_bootstrap.cc:492] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.904148 909 ts_tablet_manager.cc:1398] T f468589b44464314a74d96dd6092603d P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.904291 909 tablet_bootstrap.cc:492] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.904184 910 ts_tablet_manager.cc:1398] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.904721 910 tablet_bootstrap.cc:492] T 1cdacda459cc4daaad6fcde6f07058d5 P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.905180 909 tablet_bootstrap.cc:654] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.905262 911 tablet_bootstrap.cc:654] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.905695 910 tablet_bootstrap.cc:654] T 1cdacda459cc4daaad6fcde6f07058d5 P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.906877 911 tablet_bootstrap.cc:492] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.906952 911 ts_tablet_manager.cc:1367] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:09.907012 909 tablet_bootstrap.cc:492] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.907070 909 ts_tablet_manager.cc:1367] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:09.907326 911 raft_consensus.cc:348] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.907438 911 raft_consensus.cc:374] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.907470 911 raft_consensus.cc:729] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.907434 909 raft_consensus.cc:348] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.907510 909 raft_consensus.cc:374] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.907537 909 raft_consensus.cc:729] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.907569 911 consensus_queue.cc:260] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.907636 909 consensus_queue.cc:260] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.907860 909 ts_tablet_manager.cc:1398] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.907876 911 ts_tablet_manager.cc:1398] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.908030 911 tablet_bootstrap.cc:492] T ae1c471e7c9d4965b500bbcfcd0e0922 P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.908219 909 tablet_bootstrap.cc:492] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.908861 910 tablet_bootstrap.cc:492] T 1cdacda459cc4daaad6fcde6f07058d5 P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.908929 910 ts_tablet_manager.cc:1367] T 1cdacda459cc4daaad6fcde6f07058d5 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 07:59:09.908972 909 tablet_bootstrap.cc:654] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.909080 911 tablet_bootstrap.cc:654] T ae1c471e7c9d4965b500bbcfcd0e0922 P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.909315 910 raft_consensus.cc:348] T 1cdacda459cc4daaad6fcde6f07058d5 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.909399 910 raft_consensus.cc:374] T 1cdacda459cc4daaad6fcde6f07058d5 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.909427 910 raft_consensus.cc:729] T 1cdacda459cc4daaad6fcde6f07058d5 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.909524 910 consensus_queue.cc:260] T 1cdacda459cc4daaad6fcde6f07058d5 P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.909876 910 ts_tablet_manager.cc:1398] T 1cdacda459cc4daaad6fcde6f07058d5 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.909976 909 tablet_bootstrap.cc:492] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.910053 909 ts_tablet_manager.cc:1367] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:09.910132 910 tablet_bootstrap.cc:492] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.910414 909 raft_consensus.cc:348] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.910509 909 raft_consensus.cc:374] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.910526 911 tablet_bootstrap.cc:492] T ae1c471e7c9d4965b500bbcfcd0e0922 P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.910545 909 raft_consensus.cc:729] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.910588 911 ts_tablet_manager.cc:1367] T ae1c471e7c9d4965b500bbcfcd0e0922 P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:09.910660 909 consensus_queue.cc:260] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.910894 909 ts_tablet_manager.cc:1398] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:09.910948 911 raft_consensus.cc:348] T ae1c471e7c9d4965b500bbcfcd0e0922 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.911036 909 tablet_bootstrap.cc:492] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.911051 911 raft_consensus.cc:374] T ae1c471e7c9d4965b500bbcfcd0e0922 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.911090 911 raft_consensus.cc:729] T ae1c471e7c9d4965b500bbcfcd0e0922 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.911202 911 consensus_queue.cc:260] T ae1c471e7c9d4965b500bbcfcd0e0922 P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.911455 911 ts_tablet_manager.cc:1398] T ae1c471e7c9d4965b500bbcfcd0e0922 P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.911475 910 tablet_bootstrap.cc:654] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.911602 911 tablet_bootstrap.cc:492] T 5b2abd6fa45746aebd6eba231483ba4f P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.911842 909 tablet_bootstrap.cc:654] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.912544 911 tablet_bootstrap.cc:654] T 5b2abd6fa45746aebd6eba231483ba4f P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.912817 909 tablet_bootstrap.cc:492] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.912909 909 ts_tablet_manager.cc:1367] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:09.913480 909 raft_consensus.cc:348] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.913560 909 raft_consensus.cc:374] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.913600 909 raft_consensus.cc:729] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.913705 909 consensus_queue.cc:260] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.913749 910 tablet_bootstrap.cc:492] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.913816 910 ts_tablet_manager.cc:1367] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 07:59:09.913921 909 ts_tablet_manager.cc:1398] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.914055 909 tablet_bootstrap.cc:492] T fe588e82efae459abbe85693d4f77c14 P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.914191 910 raft_consensus.cc:348] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.914278 910 raft_consensus.cc:374] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.914312 910 raft_consensus.cc:729] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.914433 910 consensus_queue.cc:260] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.914605 911 tablet_bootstrap.cc:492] T 5b2abd6fa45746aebd6eba231483ba4f P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.914669 911 ts_tablet_manager.cc:1367] T 5b2abd6fa45746aebd6eba231483ba4f P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:09.914729 910 ts_tablet_manager.cc:1398] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.914793 909 tablet_bootstrap.cc:654] T fe588e82efae459abbe85693d4f77c14 P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.914872 910 tablet_bootstrap.cc:492] T 51a1bbf6154345b482f7425a929a01cb P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.915052 911 raft_consensus.cc:348] T 5b2abd6fa45746aebd6eba231483ba4f P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.915141 911 raft_consensus.cc:374] T 5b2abd6fa45746aebd6eba231483ba4f P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.915171 911 raft_consensus.cc:729] T 5b2abd6fa45746aebd6eba231483ba4f P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.915283 911 consensus_queue.cc:260] T 5b2abd6fa45746aebd6eba231483ba4f P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.915511 911 ts_tablet_manager.cc:1398] T 5b2abd6fa45746aebd6eba231483ba4f P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.915659 910 tablet_bootstrap.cc:654] T 51a1bbf6154345b482f7425a929a01cb P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.915673 911 tablet_bootstrap.cc:492] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.915977 909 tablet_bootstrap.cc:492] T fe588e82efae459abbe85693d4f77c14 P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.916044 909 ts_tablet_manager.cc:1367] T fe588e82efae459abbe85693d4f77c14 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:09.916487 911 tablet_bootstrap.cc:654] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.916486 909 raft_consensus.cc:348] T fe588e82efae459abbe85693d4f77c14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.916560 909 raft_consensus.cc:374] T fe588e82efae459abbe85693d4f77c14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.916599 909 raft_consensus.cc:729] T fe588e82efae459abbe85693d4f77c14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.916702 909 consensus_queue.cc:260] T fe588e82efae459abbe85693d4f77c14 P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.916926 909 ts_tablet_manager.cc:1398] T fe588e82efae459abbe85693d4f77c14 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.917060 909 tablet_bootstrap.cc:492] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.917068 910 tablet_bootstrap.cc:492] T 51a1bbf6154345b482f7425a929a01cb P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.917136 910 ts_tablet_manager.cc:1367] T 51a1bbf6154345b482f7425a929a01cb P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.917495 910 raft_consensus.cc:348] T 51a1bbf6154345b482f7425a929a01cb P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.917603 910 raft_consensus.cc:374] T 51a1bbf6154345b482f7425a929a01cb P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.917632 910 raft_consensus.cc:729] T 51a1bbf6154345b482f7425a929a01cb P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.917739 910 consensus_queue.cc:260] T 51a1bbf6154345b482f7425a929a01cb P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.917804 909 tablet_bootstrap.cc:654] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.917901 911 tablet_bootstrap.cc:492] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.917971 911 ts_tablet_manager.cc:1367] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:09.917973 910 ts_tablet_manager.cc:1398] T 51a1bbf6154345b482f7425a929a01cb P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.918118 910 tablet_bootstrap.cc:492] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.918355 911 raft_consensus.cc:348] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.918459 911 raft_consensus.cc:374] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.918488 911 raft_consensus.cc:729] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.918891 910 tablet_bootstrap.cc:654] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.919413 909 tablet_bootstrap.cc:492] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.919484 909 ts_tablet_manager.cc:1367] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.919729 911 consensus_queue.cc:260] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.919996 909 raft_consensus.cc:348] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.920027 911 ts_tablet_manager.cc:1398] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.920083 909 raft_consensus.cc:374] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.920114 909 raft_consensus.cc:729] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.920179 911 tablet_bootstrap.cc:492] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.920202 909 consensus_queue.cc:260] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.920421 909 ts_tablet_manager.cc:1398] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.920567 909 tablet_bootstrap.cc:492] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.921357 909 tablet_bootstrap.cc:654] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.922335 911 tablet_bootstrap.cc:654] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.922463 909 tablet_bootstrap.cc:492] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.922542 909 ts_tablet_manager.cc:1367] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.922919 909 raft_consensus.cc:348] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.923019 909 raft_consensus.cc:374] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.923066 909 raft_consensus.cc:729] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.923158 909 consensus_queue.cc:260] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.923381 909 ts_tablet_manager.cc:1398] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.923393 911 tablet_bootstrap.cc:492] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.923393 910 tablet_bootstrap.cc:492] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.923460 910 ts_tablet_manager.cc:1367] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 07:59:09.923460 911 ts_tablet_manager.cc:1367] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:09.923516 909 tablet_bootstrap.cc:492] T 917d5a979ab44260838db6484408c453 P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.923830 910 raft_consensus.cc:348] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.923830 911 raft_consensus.cc:348] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.923913 911 raft_consensus.cc:374] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.923974 911 raft_consensus.cc:729] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.923913 910 raft_consensus.cc:374] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.924059 910 raft_consensus.cc:729] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.924072 911 consensus_queue.cc:260] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.924167 910 consensus_queue.cc:260] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.924330 909 tablet_bootstrap.cc:654] T 917d5a979ab44260838db6484408c453 P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.924376 910 ts_tablet_manager.cc:1398] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:09.924499 910 tablet_bootstrap.cc:492] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.925143 911 ts_tablet_manager.cc:1398] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 07:59:09.925236 910 tablet_bootstrap.cc:654] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.925264 909 tablet_bootstrap.cc:492] T 917d5a979ab44260838db6484408c453 P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.925292 911 tablet_bootstrap.cc:492] T 7dd94d49fbdf4dd695057492c52ec589 P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.925329 909 ts_tablet_manager.cc:1367] T 917d5a979ab44260838db6484408c453 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.925724 909 raft_consensus.cc:348] T 917d5a979ab44260838db6484408c453 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.925814 909 raft_consensus.cc:374] T 917d5a979ab44260838db6484408c453 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.925845 909 raft_consensus.cc:729] T 917d5a979ab44260838db6484408c453 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.925956 909 consensus_queue.cc:260] T 917d5a979ab44260838db6484408c453 P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.926198 911 tablet_bootstrap.cc:654] T 7dd94d49fbdf4dd695057492c52ec589 P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.926209 909 ts_tablet_manager.cc:1398] T 917d5a979ab44260838db6484408c453 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.926348 909 tablet_bootstrap.cc:492] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.927296 909 tablet_bootstrap.cc:654] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.927655 924 raft_consensus.cc:482] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:09.927978 924 raft_consensus.cc:504] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.928779 909 tablet_bootstrap.cc:492] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.928841 909 ts_tablet_manager.cc:1367] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 07:59:09.929147 924 leader_election.cc:288] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:09.929190 909 raft_consensus.cc:348] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.929261 909 raft_consensus.cc:374] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.929288 909 raft_consensus.cc:729] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.929378 909 consensus_queue.cc:260] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.930096 909 ts_tablet_manager.cc:1398] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:09.930297 909 tablet_bootstrap.cc:492] T ae1c471e7c9d4965b500bbcfcd0e0922 P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.931237 909 tablet_bootstrap.cc:654] T ae1c471e7c9d4965b500bbcfcd0e0922 P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.931370 911 tablet_bootstrap.cc:492] T 7dd94d49fbdf4dd695057492c52ec589 P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.931413 910 tablet_bootstrap.cc:492] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.931455 911 ts_tablet_manager.cc:1367] T 7dd94d49fbdf4dd695057492c52ec589 P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240501 07:59:09.931478 910 ts_tablet_manager.cc:1367] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240501 07:59:09.932006 910 raft_consensus.cc:348] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.931901 911 raft_consensus.cc:348] T 7dd94d49fbdf4dd695057492c52ec589 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.932101 910 raft_consensus.cc:374] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.932132 910 raft_consensus.cc:729] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.932137 911 raft_consensus.cc:374] T 7dd94d49fbdf4dd695057492c52ec589 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.932169 911 raft_consensus.cc:729] T 7dd94d49fbdf4dd695057492c52ec589 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.932235 910 consensus_queue.cc:260] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.932265 911 consensus_queue.cc:260] T 7dd94d49fbdf4dd695057492c52ec589 P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.932492 910 ts_tablet_manager.cc:1398] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:09.932497 911 ts_tablet_manager.cc:1398] T 7dd94d49fbdf4dd695057492c52ec589 P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.932649 911 tablet_bootstrap.cc:492] T 2ceab1e690b845fd974f0f7dd563c768 P 1e63aa3297ad4acd8856e6aa28008957: Bootstrap starting.
I20240501 07:59:09.932653 910 tablet_bootstrap.cc:492] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.932705 909 tablet_bootstrap.cc:492] T ae1c471e7c9d4965b500bbcfcd0e0922 P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.932783 909 ts_tablet_manager.cc:1367] T ae1c471e7c9d4965b500bbcfcd0e0922 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:09.933162 909 raft_consensus.cc:348] T ae1c471e7c9d4965b500bbcfcd0e0922 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.933245 909 raft_consensus.cc:374] T ae1c471e7c9d4965b500bbcfcd0e0922 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.933274 909 raft_consensus.cc:729] T ae1c471e7c9d4965b500bbcfcd0e0922 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.933369 909 consensus_queue.cc:260] T ae1c471e7c9d4965b500bbcfcd0e0922 P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.933423 910 tablet_bootstrap.cc:654] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.933423 911 tablet_bootstrap.cc:654] T 2ceab1e690b845fd974f0f7dd563c768 P 1e63aa3297ad4acd8856e6aa28008957: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.933615 909 ts_tablet_manager.cc:1398] T ae1c471e7c9d4965b500bbcfcd0e0922 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.933758 909 tablet_bootstrap.cc:492] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.934628 909 tablet_bootstrap.cc:654] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.935765 909 tablet_bootstrap.cc:492] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.935837 909 ts_tablet_manager.cc:1367] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.936234 909 raft_consensus.cc:348] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.936316 909 raft_consensus.cc:374] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.936347 909 raft_consensus.cc:729] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.936441 909 consensus_queue.cc:260] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.936663 909 ts_tablet_manager.cc:1398] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:09.936794 909 tablet_bootstrap.cc:492] T b69525d893854177832cf2e3f8066170 P 6c6e5fc1b40e4015a97c69fef00a4cff: Bootstrap starting.
I20240501 07:59:09.937541 909 tablet_bootstrap.cc:654] T b69525d893854177832cf2e3f8066170 P 6c6e5fc1b40e4015a97c69fef00a4cff: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.938122 845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29a7d6078f164c7085e694de0fa3b490" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:09.938334 845 raft_consensus.cc:2454] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:09.938517 909 tablet_bootstrap.cc:492] T b69525d893854177832cf2e3f8066170 P 6c6e5fc1b40e4015a97c69fef00a4cff: No bootstrap required, opened a new log
I20240501 07:59:09.938580 909 ts_tablet_manager.cc:1367] T b69525d893854177832cf2e3f8066170 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 07:59:09.938694 650 leader_election.cc:302] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:09.938823 583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29a7d6078f164c7085e694de0fa3b490" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957" is_pre_election: true
I20240501 07:59:09.938952 954 raft_consensus.cc:2781] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:09.938936 909 raft_consensus.cc:348] T b69525d893854177832cf2e3f8066170 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.939004 583 raft_consensus.cc:2454] T 29a7d6078f164c7085e694de0fa3b490 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:09.939031 954 raft_consensus.cc:482] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:09.939038 909 raft_consensus.cc:374] T b69525d893854177832cf2e3f8066170 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.939101 954 raft_consensus.cc:3037] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:09.939107 909 raft_consensus.cc:729] T b69525d893854177832cf2e3f8066170 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.939214 909 consensus_queue.cc:260] T b69525d893854177832cf2e3f8066170 P 6c6e5fc1b40e4015a97c69fef00a4cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.939426 909 ts_tablet_manager.cc:1398] T b69525d893854177832cf2e3f8066170 P 6c6e5fc1b40e4015a97c69fef00a4cff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:09.940294 910 tablet_bootstrap.cc:492] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.940367 910 ts_tablet_manager.cc:1367] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240501 07:59:09.940451 911 tablet_bootstrap.cc:492] T 2ceab1e690b845fd974f0f7dd563c768 P 1e63aa3297ad4acd8856e6aa28008957: No bootstrap required, opened a new log
I20240501 07:59:09.940521 911 ts_tablet_manager.cc:1367] T 2ceab1e690b845fd974f0f7dd563c768 P 1e63aa3297ad4acd8856e6aa28008957: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240501 07:59:09.940691 954 raft_consensus.cc:504] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.940727 910 raft_consensus.cc:348] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.940835 910 raft_consensus.cc:374] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.940861 910 raft_consensus.cc:729] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.940953 910 consensus_queue.cc:260] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.940915 911 raft_consensus.cc:348] T 2ceab1e690b845fd974f0f7dd563c768 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.941012 954 leader_election.cc:288] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Requested vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:09.941064 911 raft_consensus.cc:374] T 2ceab1e690b845fd974f0f7dd563c768 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.941097 911 raft_consensus.cc:729] T 2ceab1e690b845fd974f0f7dd563c768 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.941176 910 ts_tablet_manager.cc:1398] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:09.941308 910 tablet_bootstrap.cc:492] T b69525d893854177832cf2e3f8066170 P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.941646 583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29a7d6078f164c7085e694de0fa3b490" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957"
I20240501 07:59:09.941740 583 raft_consensus.cc:3037] T 29a7d6078f164c7085e694de0fa3b490 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:09.942070 910 tablet_bootstrap.cc:654] T b69525d893854177832cf2e3f8066170 P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
W20240501 07:59:09.942507 630 tablet.cc:2369] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 07:59:09.941200 911 consensus_queue.cc:260] T 2ceab1e690b845fd974f0f7dd563c768 P 1e63aa3297ad4acd8856e6aa28008957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.942909 911 ts_tablet_manager.cc:1398] T 2ceab1e690b845fd974f0f7dd563c768 P 1e63aa3297ad4acd8856e6aa28008957: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.942945 845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29a7d6078f164c7085e694de0fa3b490" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:09.943032 845 raft_consensus.cc:3037] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:09.944379 910 tablet_bootstrap.cc:492] T b69525d893854177832cf2e3f8066170 P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.944454 910 ts_tablet_manager.cc:1367] T b69525d893854177832cf2e3f8066170 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:09.944818 910 raft_consensus.cc:348] T b69525d893854177832cf2e3f8066170 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.944900 910 raft_consensus.cc:374] T b69525d893854177832cf2e3f8066170 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.944921 910 raft_consensus.cc:729] T b69525d893854177832cf2e3f8066170 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.945008 910 consensus_queue.cc:260] T b69525d893854177832cf2e3f8066170 P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.945223 910 ts_tablet_manager.cc:1398] T b69525d893854177832cf2e3f8066170 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:09.945407 910 tablet_bootstrap.cc:492] T 5b2abd6fa45746aebd6eba231483ba4f P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.946130 910 tablet_bootstrap.cc:654] T 5b2abd6fa45746aebd6eba231483ba4f P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.947027 910 tablet_bootstrap.cc:492] T 5b2abd6fa45746aebd6eba231483ba4f P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.947088 910 ts_tablet_manager.cc:1367] T 5b2abd6fa45746aebd6eba231483ba4f P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.947407 910 raft_consensus.cc:348] T 5b2abd6fa45746aebd6eba231483ba4f P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.947499 910 raft_consensus.cc:374] T 5b2abd6fa45746aebd6eba231483ba4f P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.947531 910 raft_consensus.cc:729] T 5b2abd6fa45746aebd6eba231483ba4f P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.947649 910 consensus_queue.cc:260] T 5b2abd6fa45746aebd6eba231483ba4f P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.947798 925 raft_consensus.cc:482] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:09.947871 925 raft_consensus.cc:504] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.947916 910 ts_tablet_manager.cc:1398] T 5b2abd6fa45746aebd6eba231483ba4f P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:09.948249 910 tablet_bootstrap.cc:492] T 917d5a979ab44260838db6484408c453 P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.948235 925 leader_election.cc:288] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:09.948779 844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b69525d893854177832cf2e3f8066170" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:09.948848 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b69525d893854177832cf2e3f8066170" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" is_pre_election: true
I20240501 07:59:09.948889 844 raft_consensus.cc:2454] T b69525d893854177832cf2e3f8066170 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 0.
I20240501 07:59:09.948949 910 tablet_bootstrap.cc:654] T 917d5a979ab44260838db6484408c453 P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.948973 714 raft_consensus.cc:2454] T b69525d893854177832cf2e3f8066170 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 0.
I20240501 07:59:09.949240 519 leader_election.cc:302] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 6c6e5fc1b40e4015a97c69fef00a4cff; no voters:
I20240501 07:59:09.949419 925 raft_consensus.cc:2781] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:09.949469 925 raft_consensus.cc:482] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:09.949506 925 raft_consensus.cc:3037] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:09.949813 910 tablet_bootstrap.cc:492] T 917d5a979ab44260838db6484408c453 P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.949874 910 ts_tablet_manager.cc:1367] T 917d5a979ab44260838db6484408c453 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:09.950170 910 raft_consensus.cc:348] T 917d5a979ab44260838db6484408c453 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.950239 910 raft_consensus.cc:374] T 917d5a979ab44260838db6484408c453 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.950269 910 raft_consensus.cc:729] T 917d5a979ab44260838db6484408c453 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.950371 910 consensus_queue.cc:260] T 917d5a979ab44260838db6484408c453 P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.950559 910 ts_tablet_manager.cc:1398] T 917d5a979ab44260838db6484408c453 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:09.950691 910 tablet_bootstrap.cc:492] T 606e2456198346be96fcb6b9e71f78ad P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.951377 910 tablet_bootstrap.cc:654] T 606e2456198346be96fcb6b9e71f78ad P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.952711 910 tablet_bootstrap.cc:492] T 606e2456198346be96fcb6b9e71f78ad P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.952778 910 ts_tablet_manager.cc:1367] T 606e2456198346be96fcb6b9e71f78ad P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 07:59:09.953109 910 raft_consensus.cc:348] T 606e2456198346be96fcb6b9e71f78ad P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.953198 910 raft_consensus.cc:374] T 606e2456198346be96fcb6b9e71f78ad P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.953239 910 raft_consensus.cc:729] T 606e2456198346be96fcb6b9e71f78ad P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.953222 919 raft_consensus.cc:482] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:09.953323 919 raft_consensus.cc:504] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.953357 910 consensus_queue.cc:260] T 606e2456198346be96fcb6b9e71f78ad P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.953644 910 ts_tablet_manager.cc:1398] T 606e2456198346be96fcb6b9e71f78ad P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:09.953806 910 tablet_bootstrap.cc:492] T fe588e82efae459abbe85693d4f77c14 P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.954432 919 leader_election.cc:288] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793)
I20240501 07:59:09.954545 910 tablet_bootstrap.cc:654] T fe588e82efae459abbe85693d4f77c14 P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.955492 910 tablet_bootstrap.cc:492] T fe588e82efae459abbe85693d4f77c14 P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.955578 910 ts_tablet_manager.cc:1367] T fe588e82efae459abbe85693d4f77c14 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 07:59:09.955950 910 raft_consensus.cc:348] T fe588e82efae459abbe85693d4f77c14 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.956033 910 raft_consensus.cc:374] T fe588e82efae459abbe85693d4f77c14 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.956060 910 raft_consensus.cc:729] T fe588e82efae459abbe85693d4f77c14 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.956141 910 consensus_queue.cc:260] T fe588e82efae459abbe85693d4f77c14 P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.956367 910 ts_tablet_manager.cc:1398] T fe588e82efae459abbe85693d4f77c14 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:09.956506 910 tablet_bootstrap.cc:492] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb: Bootstrap starting.
I20240501 07:59:09.956903 924 raft_consensus.cc:482] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:09.957192 910 tablet_bootstrap.cc:654] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:09.956977 924 raft_consensus.cc:504] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.958146 924 leader_election.cc:288] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:09.958446 582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2abd6fa45746aebd6eba231483ba4f" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957" is_pre_election: true
I20240501 07:59:09.958482 844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2abd6fa45746aebd6eba231483ba4f" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:09.958570 582 raft_consensus.cc:2454] T 5b2abd6fa45746aebd6eba231483ba4f P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:09.958580 844 raft_consensus.cc:2454] T 5b2abd6fa45746aebd6eba231483ba4f P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:09.958765 581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1c471e7c9d4965b500bbcfcd0e0922" candidate_uuid: "8a1a8654e29140f184ff9da4d55aceeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957" is_pre_election: true
I20240501 07:59:09.958859 581 raft_consensus.cc:2454] T ae1c471e7c9d4965b500bbcfcd0e0922 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a1a8654e29140f184ff9da4d55aceeb in term 0.
I20240501 07:59:09.958906 650 leader_election.cc:302] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:09.959158 924 raft_consensus.cc:2781] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:09.959219 924 raft_consensus.cc:482] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:09.959264 924 raft_consensus.cc:3037] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:09.959338 780 leader_election.cc:302] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:09.959550 919 raft_consensus.cc:2781] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:09.959594 919 raft_consensus.cc:482] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:09.959627 919 raft_consensus.cc:3037] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:09.960201 913 raft_consensus.cc:482] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:09.960271 913 raft_consensus.cc:504] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.960573 913 leader_election.cc:288] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:09.961189 910 tablet_bootstrap.cc:492] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb: No bootstrap required, opened a new log
I20240501 07:59:09.961258 910 ts_tablet_manager.cc:1367] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240501 07:59:09.961338 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe588e82efae459abbe85693d4f77c14" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" is_pre_election: true
I20240501 07:59:09.961422 713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1c471e7c9d4965b500bbcfcd0e0922" candidate_uuid: "8a1a8654e29140f184ff9da4d55aceeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" is_pre_election: true
I20240501 07:59:09.961455 714 raft_consensus.cc:2454] T fe588e82efae459abbe85693d4f77c14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 0.
I20240501 07:59:09.961506 713 raft_consensus.cc:2454] T ae1c471e7c9d4965b500bbcfcd0e0922 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a1a8654e29140f184ff9da4d55aceeb in term 0.
I20240501 07:59:09.961256 844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe588e82efae459abbe85693d4f77c14" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:09.961668 844 raft_consensus.cc:2454] T fe588e82efae459abbe85693d4f77c14 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 0.
I20240501 07:59:09.961640 910 raft_consensus.cc:348] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.961723 910 raft_consensus.cc:374] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:09.961753 910 raft_consensus.cc:729] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Initialized, Role: FOLLOWER
I20240501 07:59:09.961827 519 leader_election.cc:302] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 6c6e5fc1b40e4015a97c69fef00a4cff; no voters:
I20240501 07:59:09.961851 910 consensus_queue.cc:260] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:09.961975 913 raft_consensus.cc:2781] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:09.962029 913 raft_consensus.cc:482] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:09.962072 913 raft_consensus.cc:3037] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:09.962306 910 ts_tablet_manager.cc:1398] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:10.014039 954 raft_consensus.cc:482] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:10.014164 954 raft_consensus.cc:504] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.014598 954 leader_election.cc:288] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.015036 844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "606e2456198346be96fcb6b9e71f78ad" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:10.015122 581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "606e2456198346be96fcb6b9e71f78ad" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957" is_pre_election: true
I20240501 07:59:10.015269 581 raft_consensus.cc:2454] T 606e2456198346be96fcb6b9e71f78ad P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:10.015254 844 raft_consensus.cc:2454] T 606e2456198346be96fcb6b9e71f78ad P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:10.015592 650 leader_election.cc:302] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.015780 954 raft_consensus.cc:2781] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:10.015841 954 raft_consensus.cc:482] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:10.015866 954 raft_consensus.cc:3037] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.024983 972 raft_consensus.cc:482] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:10.025079 972 raft_consensus.cc:504] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.025434 972 leader_election.cc:288] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.025804 713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51a1bbf6154345b482f7425a929a01cb" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" is_pre_election: true
I20240501 07:59:10.025840 844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51a1bbf6154345b482f7425a929a01cb" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:10.025921 713 raft_consensus.cc:2454] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 0.
I20240501 07:59:10.025955 844 raft_consensus.cc:2454] T 51a1bbf6154345b482f7425a929a01cb P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 0.
I20240501 07:59:10.026176 519 leader_election.cc:302] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 6c6e5fc1b40e4015a97c69fef00a4cff; no voters:
I20240501 07:59:10.026343 972 raft_consensus.cc:2781] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:10.026394 972 raft_consensus.cc:482] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:10.026436 972 raft_consensus.cc:3037] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
W20240501 07:59:10.031801 892 tablet.cc:2369] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 07:59:10.040673 845 raft_consensus.cc:2454] T 29a7d6078f164c7085e694de0fa3b490 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.040679 583 raft_consensus.cc:2454] T 29a7d6078f164c7085e694de0fa3b490 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.041039 650 leader_election.cc:302] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.041078 924 raft_consensus.cc:504] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.041433 924 leader_election.cc:288] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Requested vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.041523 924 raft_consensus.cc:2781] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.041579 924 raft_consensus.cc:686] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 LEADER]: Becoming Leader. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Running, Role: LEADER
I20240501 07:59:10.041846 925 raft_consensus.cc:504] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.041893 924 consensus_queue.cc:237] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.042196 925 leader_election.cc:288] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 election: Requested vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.043001 919 raft_consensus.cc:504] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.043141 954 raft_consensus.cc:504] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.043318 919 leader_election.cc:288] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [CANDIDATE]: Term 1 election: Requested vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793)
I20240501 07:59:10.043431 954 leader_election.cc:288] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Requested vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.044174 845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2abd6fa45746aebd6eba231483ba4f" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:10.044278 845 raft_consensus.cc:3037] T 5b2abd6fa45746aebd6eba231483ba4f P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.044643 844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "606e2456198346be96fcb6b9e71f78ad" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:10.044745 844 raft_consensus.cc:3037] T 606e2456198346be96fcb6b9e71f78ad P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.045063 713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1c471e7c9d4965b500bbcfcd0e0922" candidate_uuid: "8a1a8654e29140f184ff9da4d55aceeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff"
I20240501 07:59:10.045151 713 raft_consensus.cc:3037] T ae1c471e7c9d4965b500bbcfcd0e0922 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.045644 843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b69525d893854177832cf2e3f8066170" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:10.045743 843 raft_consensus.cc:3037] T b69525d893854177832cf2e3f8066170 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.046036 583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae1c471e7c9d4965b500bbcfcd0e0922" candidate_uuid: "8a1a8654e29140f184ff9da4d55aceeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957"
I20240501 07:59:10.046144 581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2abd6fa45746aebd6eba231483ba4f" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957"
I20240501 07:59:10.046217 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b69525d893854177832cf2e3f8066170" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff"
I20240501 07:59:10.046249 581 raft_consensus.cc:3037] T 5b2abd6fa45746aebd6eba231483ba4f P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.046305 714 raft_consensus.cc:3037] T b69525d893854177832cf2e3f8066170 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.046144 583 raft_consensus.cc:3037] T ae1c471e7c9d4965b500bbcfcd0e0922 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.046455 582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "606e2456198346be96fcb6b9e71f78ad" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957"
I20240501 07:59:10.046526 582 raft_consensus.cc:3037] T 606e2456198346be96fcb6b9e71f78ad P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.048529 439 catalog_manager.cc:5484] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130). New cstate: current_term: 1 leader_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.063913 924 raft_consensus.cc:482] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:10.064047 924 raft_consensus.cc:504] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.064396 924 leader_election.cc:288] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.064791 580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befa9b316ce74132882efeee4bb05aa1" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957" is_pre_election: true
I20240501 07:59:10.064821 842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befa9b316ce74132882efeee4bb05aa1" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:10.064945 842 raft_consensus.cc:2454] T befa9b316ce74132882efeee4bb05aa1 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:10.065057 580 raft_consensus.cc:2454] T befa9b316ce74132882efeee4bb05aa1 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:10.065215 650 leader_election.cc:302] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.065382 924 raft_consensus.cc:2781] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:10.065438 924 raft_consensus.cc:482] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:10.065475 924 raft_consensus.cc:3037] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.081380 919 raft_consensus.cc:482] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:10.081488 919 raft_consensus.cc:504] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.081857 919 leader_election.cc:288] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793)
I20240501 07:59:10.082288 580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07cd57b9e80a4231944fc0d91f363c99" candidate_uuid: "8a1a8654e29140f184ff9da4d55aceeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957" is_pre_election: true
I20240501 07:59:10.082417 580 raft_consensus.cc:2454] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a1a8654e29140f184ff9da4d55aceeb in term 0.
I20240501 07:59:10.082684 780 leader_election.cc:302] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.082757 712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07cd57b9e80a4231944fc0d91f363c99" candidate_uuid: "8a1a8654e29140f184ff9da4d55aceeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" is_pre_election: true
I20240501 07:59:10.082828 919 raft_consensus.cc:2781] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:10.082870 919 raft_consensus.cc:482] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:10.082911 919 raft_consensus.cc:3037] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.082897 712 raft_consensus.cc:2454] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a1a8654e29140f184ff9da4d55aceeb in term 0.
W20240501 07:59:10.106771 761 tablet.cc:2369] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 07:59:10.129860 925 raft_consensus.cc:482] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:10.129984 925 raft_consensus.cc:504] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.130393 925 leader_election.cc:288] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.130934 842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cdacda459cc4daaad6fcde6f07058d5" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:10.131060 842 raft_consensus.cc:2454] T 1cdacda459cc4daaad6fcde6f07058d5 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 0.
I20240501 07:59:10.131142 712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cdacda459cc4daaad6fcde6f07058d5" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" is_pre_election: true
I20240501 07:59:10.131285 712 raft_consensus.cc:2454] T 1cdacda459cc4daaad6fcde6f07058d5 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 0.
I20240501 07:59:10.131364 519 leader_election.cc:302] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.131578 925 raft_consensus.cc:2781] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:10.131641 925 raft_consensus.cc:482] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:10.131690 925 raft_consensus.cc:3037] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.140405 713 raft_consensus.cc:2454] T ae1c471e7c9d4965b500bbcfcd0e0922 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a1a8654e29140f184ff9da4d55aceeb in term 1.
I20240501 07:59:10.140408 913 raft_consensus.cc:504] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.140790 913 leader_election.cc:288] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 election: Requested vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.140810 781 leader_election.cc:302] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.141019 967 raft_consensus.cc:2781] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.141127 713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe588e82efae459abbe85693d4f77c14" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff"
I20240501 07:59:10.141140 842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe588e82efae459abbe85693d4f77c14" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:10.141239 842 raft_consensus.cc:3037] T fe588e82efae459abbe85693d4f77c14 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.141242 713 raft_consensus.cc:3037] T fe588e82efae459abbe85693d4f77c14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.141268 967 raft_consensus.cc:686] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 LEADER]: Becoming Leader. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Running, Role: LEADER
I20240501 07:59:10.141619 967 consensus_queue.cc:237] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.146282 973 raft_consensus.cc:482] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:10.146361 973 raft_consensus.cc:504] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.146660 973 leader_election.cc:288] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.146971 841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2382ca3cb43a4cff9b4594a9936ac9e9" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:10.146992 580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2382ca3cb43a4cff9b4594a9936ac9e9" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957" is_pre_election: true
I20240501 07:59:10.147065 841 raft_consensus.cc:2454] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:10.147104 580 raft_consensus.cc:2454] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:10.147316 650 leader_election.cc:302] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.147475 973 raft_consensus.cc:2781] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:10.147528 973 raft_consensus.cc:482] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:10.147578 973 raft_consensus.cc:3037] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.149681 845 raft_consensus.cc:2454] T 5b2abd6fa45746aebd6eba231483ba4f P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.150002 650 leader_election.cc:302] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.150264 972 raft_consensus.cc:504] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.150446 581 raft_consensus.cc:2454] T 5b2abd6fa45746aebd6eba231483ba4f P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.150506 954 raft_consensus.cc:2781] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.150621 972 leader_election.cc:288] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 election: Requested vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.150698 954 raft_consensus.cc:686] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 LEADER]: Becoming Leader. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Running, Role: LEADER
I20240501 07:59:10.150799 844 raft_consensus.cc:2454] T 606e2456198346be96fcb6b9e71f78ad P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.151594 583 raft_consensus.cc:2454] T ae1c471e7c9d4965b500bbcfcd0e0922 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a1a8654e29140f184ff9da4d55aceeb in term 1.
I20240501 07:59:10.151618 843 raft_consensus.cc:2454] T b69525d893854177832cf2e3f8066170 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 1.
I20240501 07:59:10.151829 954 consensus_queue.cc:237] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.152156 582 raft_consensus.cc:2454] T 606e2456198346be96fcb6b9e71f78ad P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.152413 919 raft_consensus.cc:504] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.152717 714 raft_consensus.cc:2454] T b69525d893854177832cf2e3f8066170 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 1.
I20240501 07:59:10.152731 919 leader_election.cc:288] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [CANDIDATE]: Term 1 election: Requested vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793)
I20240501 07:59:10.152828 925 raft_consensus.cc:504] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.153092 650 leader_election.cc:302] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.153106 925 leader_election.cc:288] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 election: Requested vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.153278 977 raft_consensus.cc:2781] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.153306 924 raft_consensus.cc:504] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.153412 977 raft_consensus.cc:686] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 LEADER]: Becoming Leader. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Running, Role: LEADER
I20240501 07:59:10.153544 977 consensus_queue.cc:237] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.153733 924 leader_election.cc:288] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Requested vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.154512 439 catalog_manager.cc:5484] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131). New cstate: current_term: 1 leader_uuid: "8a1a8654e29140f184ff9da4d55aceeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:10.154865 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07cd57b9e80a4231944fc0d91f363c99" candidate_uuid: "8a1a8654e29140f184ff9da4d55aceeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff"
I20240501 07:59:10.154955 714 raft_consensus.cc:3037] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.155946 843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befa9b316ce74132882efeee4bb05aa1" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:10.156049 843 raft_consensus.cc:3037] T befa9b316ce74132882efeee4bb05aa1 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.156141 441 catalog_manager.cc:5484] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130). New cstate: current_term: 1 leader_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.156381 441 catalog_manager.cc:5484] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130). New cstate: current_term: 1 leader_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.156545 712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51a1bbf6154345b482f7425a929a01cb" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff"
I20240501 07:59:10.156646 712 raft_consensus.cc:3037] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.157063 844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51a1bbf6154345b482f7425a929a01cb" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:10.157162 844 raft_consensus.cc:3037] T 51a1bbf6154345b482f7425a929a01cb P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.157503 711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cdacda459cc4daaad6fcde6f07058d5" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff"
I20240501 07:59:10.157598 711 raft_consensus.cc:3037] T 1cdacda459cc4daaad6fcde6f07058d5 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.157753 845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cdacda459cc4daaad6fcde6f07058d5" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:10.157744 582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befa9b316ce74132882efeee4bb05aa1" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957"
I20240501 07:59:10.157845 845 raft_consensus.cc:3037] T 1cdacda459cc4daaad6fcde6f07058d5 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.157848 582 raft_consensus.cc:3037] T befa9b316ce74132882efeee4bb05aa1 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.158325 519 leader_election.cc:302] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.158586 925 raft_consensus.cc:2781] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.158680 925 raft_consensus.cc:686] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 LEADER]: Becoming Leader. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Running, Role: LEADER
I20240501 07:59:10.158726 583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07cd57b9e80a4231944fc0d91f363c99" candidate_uuid: "8a1a8654e29140f184ff9da4d55aceeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957"
I20240501 07:59:10.158830 583 raft_consensus.cc:3037] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.158982 925 consensus_queue.cc:237] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.161723 441 catalog_manager.cc:5484] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "1e63aa3297ad4acd8856e6aa28008957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.174978 919 raft_consensus.cc:482] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:10.175109 919 raft_consensus.cc:504] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.175477 919 leader_election.cc:288] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459)
I20240501 07:59:10.175915 581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f468589b44464314a74d96dd6092603d" candidate_uuid: "8a1a8654e29140f184ff9da4d55aceeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957" is_pre_election: true
I20240501 07:59:10.175940 710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f468589b44464314a74d96dd6092603d" candidate_uuid: "8a1a8654e29140f184ff9da4d55aceeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" is_pre_election: true
I20240501 07:59:10.176074 710 raft_consensus.cc:2454] T f468589b44464314a74d96dd6092603d P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a1a8654e29140f184ff9da4d55aceeb in term 0.
I20240501 07:59:10.176076 581 raft_consensus.cc:2454] T f468589b44464314a74d96dd6092603d P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a1a8654e29140f184ff9da4d55aceeb in term 0.
I20240501 07:59:10.176335 780 leader_election.cc:302] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.176504 919 raft_consensus.cc:2781] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:10.176561 919 raft_consensus.cc:482] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:10.176612 919 raft_consensus.cc:3037] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.199666 977 raft_consensus.cc:482] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:10.199793 977 raft_consensus.cc:504] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.200238 977 leader_election.cc:288] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.200646 841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dd94d49fbdf4dd695057492c52ec589" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:10.200789 841 raft_consensus.cc:2454] T 7dd94d49fbdf4dd695057492c52ec589 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:10.200814 581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dd94d49fbdf4dd695057492c52ec589" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957" is_pre_election: true
I20240501 07:59:10.200959 581 raft_consensus.cc:2454] T 7dd94d49fbdf4dd695057492c52ec589 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:10.201056 650 leader_election.cc:302] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.201238 977 raft_consensus.cc:2781] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:10.201283 977 raft_consensus.cc:482] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:10.201309 977 raft_consensus.cc:3037] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.207809 924 raft_consensus.cc:482] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:10.207898 924 raft_consensus.cc:504] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.208261 924 leader_election.cc:288] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.208603 841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ceab1e690b845fd974f0f7dd563c768" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:10.208612 581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ceab1e690b845fd974f0f7dd563c768" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957" is_pre_election: true
I20240501 07:59:10.208717 581 raft_consensus.cc:2454] T 2ceab1e690b845fd974f0f7dd563c768 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:10.208722 841 raft_consensus.cc:2454] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:10.208971 650 leader_election.cc:302] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.209112 924 raft_consensus.cc:2781] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:10.209172 924 raft_consensus.cc:482] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:10.209197 924 raft_consensus.cc:3037] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.212728 925 raft_consensus.cc:482] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:10.212805 925 raft_consensus.cc:504] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.213125 925 leader_election.cc:288] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.213454 710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917d5a979ab44260838db6484408c453" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" is_pre_election: true
I20240501 07:59:10.213510 841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917d5a979ab44260838db6484408c453" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:10.213565 710 raft_consensus.cc:2454] T 917d5a979ab44260838db6484408c453 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 0.
I20240501 07:59:10.213636 841 raft_consensus.cc:2454] T 917d5a979ab44260838db6484408c453 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 0.
I20240501 07:59:10.213836 519 leader_election.cc:302] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 6c6e5fc1b40e4015a97c69fef00a4cff; no voters:
I20240501 07:59:10.214008 925 raft_consensus.cc:2781] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:10.214061 925 raft_consensus.cc:482] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:10.214085 925 raft_consensus.cc:3037] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.247222 582 raft_consensus.cc:2454] T befa9b316ce74132882efeee4bb05aa1 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.247222 714 raft_consensus.cc:2454] T 07cd57b9e80a4231944fc0d91f363c99 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a1a8654e29140f184ff9da4d55aceeb in term 1.
I20240501 07:59:10.247222 842 raft_consensus.cc:2454] T fe588e82efae459abbe85693d4f77c14 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 1.
I20240501 07:59:10.247596 713 raft_consensus.cc:2454] T fe588e82efae459abbe85693d4f77c14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 1.
I20240501 07:59:10.248076 781 leader_election.cc:302] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.248311 843 raft_consensus.cc:2454] T befa9b316ce74132882efeee4bb05aa1 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.248324 583 raft_consensus.cc:2454] T 07cd57b9e80a4231944fc0d91f363c99 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a1a8654e29140f184ff9da4d55aceeb in term 1.
I20240501 07:59:10.248370 973 raft_consensus.cc:504] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.248507 967 raft_consensus.cc:2781] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.248579 967 raft_consensus.cc:686] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 LEADER]: Becoming Leader. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Running, Role: LEADER
I20240501 07:59:10.248721 967 consensus_queue.cc:237] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.249040 973 leader_election.cc:288] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Requested vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.249132 844 raft_consensus.cc:2454] T 51a1bbf6154345b482f7425a929a01cb P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 1.
I20240501 07:59:10.249934 712 raft_consensus.cc:2454] T 51a1bbf6154345b482f7425a929a01cb P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 1.
I20240501 07:59:10.249986 925 raft_consensus.cc:504] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.250257 711 raft_consensus.cc:2454] T 1cdacda459cc4daaad6fcde6f07058d5 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 1.
I20240501 07:59:10.250298 925 leader_election.cc:288] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 election: Requested vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.250671 977 raft_consensus.cc:504] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.250986 977 leader_election.cc:288] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Requested vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.251020 845 raft_consensus.cc:2454] T 1cdacda459cc4daaad6fcde6f07058d5 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 1.
I20240501 07:59:10.251397 650 leader_election.cc:302] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.251688 924 raft_consensus.cc:504] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.251701 919 raft_consensus.cc:504] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.251834 845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2382ca3cb43a4cff9b4594a9936ac9e9" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:10.251950 845 raft_consensus.cc:3037] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.252028 919 leader_election.cc:288] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [CANDIDATE]: Term 1 election: Requested vote from peers 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130:43793), 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459)
I20240501 07:59:10.252254 924 leader_election.cc:288] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Requested vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.252344 924 raft_consensus.cc:2781] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.252437 711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f468589b44464314a74d96dd6092603d" candidate_uuid: "8a1a8654e29140f184ff9da4d55aceeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff"
I20240501 07:59:10.252532 711 raft_consensus.cc:3037] T f468589b44464314a74d96dd6092603d P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.252607 924 raft_consensus.cc:686] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 LEADER]: Becoming Leader. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Running, Role: LEADER
I20240501 07:59:10.252662 843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ceab1e690b845fd974f0f7dd563c768" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:10.252743 843 raft_consensus.cc:3037] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.252754 924 consensus_queue.cc:237] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.252918 844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dd94d49fbdf4dd695057492c52ec589" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:10.252993 844 raft_consensus.cc:3037] T 7dd94d49fbdf4dd695057492c52ec589 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.253643 441 catalog_manager.cc:5484] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131). New cstate: current_term: 1 leader_uuid: "8a1a8654e29140f184ff9da4d55aceeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:10.253923 712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917d5a979ab44260838db6484408c453" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff"
I20240501 07:59:10.254009 712 raft_consensus.cc:3037] T 917d5a979ab44260838db6484408c453 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.254019 842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917d5a979ab44260838db6484408c453" candidate_uuid: "1e63aa3297ad4acd8856e6aa28008957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:10.254094 842 raft_consensus.cc:3037] T 917d5a979ab44260838db6484408c453 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.254534 583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2382ca3cb43a4cff9b4594a9936ac9e9" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957"
I20240501 07:59:10.254473 439 catalog_manager.cc:5484] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130). New cstate: current_term: 1 leader_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.254565 582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dd94d49fbdf4dd695057492c52ec589" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957"
I20240501 07:59:10.254655 582 raft_consensus.cc:3037] T 7dd94d49fbdf4dd695057492c52ec589 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.254643 583 raft_consensus.cc:3037] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.255390 581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ceab1e690b845fd974f0f7dd563c768" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957"
I20240501 07:59:10.255481 581 raft_consensus.cc:3037] T 2ceab1e690b845fd974f0f7dd563c768 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.255748 519 leader_election.cc:302] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 6c6e5fc1b40e4015a97c69fef00a4cff; no voters:
I20240501 07:59:10.255877 580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f468589b44464314a74d96dd6092603d" candidate_uuid: "8a1a8654e29140f184ff9da4d55aceeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957"
I20240501 07:59:10.255877 925 raft_consensus.cc:2781] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.256011 580 raft_consensus.cc:3037] T f468589b44464314a74d96dd6092603d P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.256027 925 raft_consensus.cc:686] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 LEADER]: Becoming Leader. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Running, Role: LEADER
I20240501 07:59:10.256145 925 consensus_queue.cc:237] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.256505 519 leader_election.cc:302] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 6c6e5fc1b40e4015a97c69fef00a4cff; no voters:
I20240501 07:59:10.256650 519 leader_election.cc:302] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 6c6e5fc1b40e4015a97c69fef00a4cff; no voters:
I20240501 07:59:10.256665 913 raft_consensus.cc:2781] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.256822 972 raft_consensus.cc:2781] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.256834 913 raft_consensus.cc:686] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [term 1 LEADER]: Becoming Leader. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Running, Role: LEADER
I20240501 07:59:10.257002 913 consensus_queue.cc:237] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.257068 972 raft_consensus.cc:686] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 LEADER]: Becoming Leader. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Running, Role: LEADER
I20240501 07:59:10.257243 972 consensus_queue.cc:237] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.258702 439 catalog_manager.cc:5484] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "1e63aa3297ad4acd8856e6aa28008957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.261132 439 catalog_manager.cc:5484] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "1e63aa3297ad4acd8856e6aa28008957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.261327 439 catalog_manager.cc:5484] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "1e63aa3297ad4acd8856e6aa28008957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.280380 924 raft_consensus.cc:482] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:10.280498 924 raft_consensus.cc:504] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.280894 924 leader_election.cc:288] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.281311 579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b3f550660c441f381045fd89143ec14" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957" is_pre_election: true
I20240501 07:59:10.281302 841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b3f550660c441f381045fd89143ec14" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb" is_pre_election: true
I20240501 07:59:10.281445 841 raft_consensus.cc:2454] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:10.281445 579 raft_consensus.cc:2454] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 0.
I20240501 07:59:10.281733 649 leader_election.cc:302] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 6c6e5fc1b40e4015a97c69fef00a4cff; no voters:
I20240501 07:59:10.281895 924 raft_consensus.cc:2781] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:10.281953 924 raft_consensus.cc:482] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:10.281978 924 raft_consensus.cc:3037] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.340174 845 raft_consensus.cc:2454] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.340174 582 raft_consensus.cc:2454] T 7dd94d49fbdf4dd695057492c52ec589 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.340174 711 raft_consensus.cc:2454] T f468589b44464314a74d96dd6092603d P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a1a8654e29140f184ff9da4d55aceeb in term 1.
I20240501 07:59:10.340549 583 raft_consensus.cc:2454] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.341044 649 leader_election.cc:302] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 6c6e5fc1b40e4015a97c69fef00a4cff; no voters:
I20240501 07:59:10.341207 712 raft_consensus.cc:2454] T 917d5a979ab44260838db6484408c453 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 1.
I20240501 07:59:10.341260 649 leader_election.cc:302] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 6c6e5fc1b40e4015a97c69fef00a4cff; no voters:
I20240501 07:59:10.341332 843 raft_consensus.cc:2454] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.341498 977 raft_consensus.cc:2781] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.341574 977 raft_consensus.cc:686] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 LEADER]: Becoming Leader. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Running, Role: LEADER
I20240501 07:59:10.341637 519 leader_election.cc:302] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 6c6e5fc1b40e4015a97c69fef00a4cff; no voters:
I20240501 07:59:10.341729 977 consensus_queue.cc:237] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.342197 581 raft_consensus.cc:2454] T 2ceab1e690b845fd974f0f7dd563c768 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.342928 924 raft_consensus.cc:504] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.341506 973 raft_consensus.cc:2781] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.343346 973 raft_consensus.cc:686] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 LEADER]: Becoming Leader. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Running, Role: LEADER
I20240501 07:59:10.343755 924 leader_election.cc:288] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Requested vote from peers 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129:36459), 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131:44687)
I20240501 07:59:10.344015 580 raft_consensus.cc:2454] T f468589b44464314a74d96dd6092603d P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a1a8654e29140f184ff9da4d55aceeb in term 1.
I20240501 07:59:10.343628 973 consensus_queue.cc:237] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.344275 925 raft_consensus.cc:2781] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.344437 925 raft_consensus.cc:686] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 LEADER]: Becoming Leader. State: Replica: 1e63aa3297ad4acd8856e6aa28008957, State: Running, Role: LEADER
I20240501 07:59:10.344714 925 consensus_queue.cc:237] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.346714 781 leader_election.cc:302] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.347182 919 raft_consensus.cc:2781] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.347302 919 raft_consensus.cc:686] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [term 1 LEADER]: Becoming Leader. State: Replica: 8a1a8654e29140f184ff9da4d55aceeb, State: Running, Role: LEADER
I20240501 07:59:10.347493 919 consensus_queue.cc:237] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.349236 650 leader_election.cc:302] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6e5fc1b40e4015a97c69fef00a4cff, 8a1a8654e29140f184ff9da4d55aceeb; no voters:
I20240501 07:59:10.349632 973 raft_consensus.cc:2781] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.349422 439 catalog_manager.cc:5484] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130). New cstate: current_term: 1 leader_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.349833 973 raft_consensus.cc:686] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 LEADER]: Becoming Leader. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Running, Role: LEADER
I20240501 07:59:10.350065 973 consensus_queue.cc:237] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.350101 439 catalog_manager.cc:5484] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130). New cstate: current_term: 1 leader_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.350896 441 catalog_manager.cc:5484] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e63aa3297ad4acd8856e6aa28008957 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "1e63aa3297ad4acd8856e6aa28008957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.351233 843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b3f550660c441f381045fd89143ec14" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1a8654e29140f184ff9da4d55aceeb"
I20240501 07:59:10.351341 843 raft_consensus.cc:3037] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.351347 580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b3f550660c441f381045fd89143ec14" candidate_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e63aa3297ad4acd8856e6aa28008957"
I20240501 07:59:10.351439 580 raft_consensus.cc:3037] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:10.352576 440 catalog_manager.cc:5484] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a1a8654e29140f184ff9da4d55aceeb (127.0.99.131). New cstate: current_term: 1 leader_uuid: "8a1a8654e29140f184ff9da4d55aceeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:10.353837 441 catalog_manager.cc:5484] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130). New cstate: current_term: 1 leader_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.433718 580 raft_consensus.cc:2454] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.433713 844 raft_consensus.cc:2454] T 7dd94d49fbdf4dd695057492c52ec589 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.433779 842 raft_consensus.cc:2454] T 917d5a979ab44260838db6484408c453 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e63aa3297ad4acd8856e6aa28008957 in term 1.
I20240501 07:59:10.434276 649 leader_election.cc:302] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e63aa3297ad4acd8856e6aa28008957, 6c6e5fc1b40e4015a97c69fef00a4cff; no voters:
I20240501 07:59:10.434489 973 raft_consensus.cc:2781] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:10.434572 973 raft_consensus.cc:686] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [term 1 LEADER]: Becoming Leader. State: Replica: 6c6e5fc1b40e4015a97c69fef00a4cff, State: Running, Role: LEADER
I20240501 07:59:10.434708 973 consensus_queue.cc:237] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } }
I20240501 07:59:10.435070 843 raft_consensus.cc:2454] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6e5fc1b40e4015a97c69fef00a4cff in term 1.
I20240501 07:59:10.436477 441 catalog_manager.cc:5484] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c6e5fc1b40e4015a97c69fef00a4cff (127.0.99.130). New cstate: current_term: 1 leader_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:10.501631 973 consensus_queue.cc:1035] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.505532 973 consensus_queue.cc:1035] T 29a7d6078f164c7085e694de0fa3b490 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.520824 919 consensus_queue.cc:1035] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.524063 919 consensus_queue.cc:1035] T ae1c471e7c9d4965b500bbcfcd0e0922 P 8a1a8654e29140f184ff9da4d55aceeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.530516 973 consensus_queue.cc:1035] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.533501 973 consensus_queue.cc:1035] T 606e2456198346be96fcb6b9e71f78ad P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.597049 972 consensus_queue.cc:1035] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.600906 972 consensus_queue.cc:1035] T b69525d893854177832cf2e3f8066170 P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.641425 925 consensus_queue.cc:1035] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.645915 984 consensus_queue.cc:1035] T 51a1bbf6154345b482f7425a929a01cb P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 07:59:10.661154 1000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:10.661381 1000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:10.664445 1000 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 07:59:10.671037 954 consensus_queue.cc:1035] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.674151 954 consensus_queue.cc:1035] T befa9b316ce74132882efeee4bb05aa1 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.686018 954 consensus_queue.cc:1035] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.688844 973 consensus_queue.cc:1035] T 5b2abd6fa45746aebd6eba231483ba4f P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_0'
I20240501 07:59:10.712698 925 consensus_queue.cc:1035] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.725662 925 consensus_queue.cc:1035] T 1cdacda459cc4daaad6fcde6f07058d5 P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.733947 975 consensus_queue.cc:1035] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.734091 973 consensus_queue.cc:1035] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.737066 984 consensus_queue.cc:1035] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.737097 967 consensus_queue.cc:1035] T 07cd57b9e80a4231944fc0d91f363c99 P 8a1a8654e29140f184ff9da4d55aceeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.739136 977 consensus_queue.cc:1035] T 7dd94d49fbdf4dd695057492c52ec589 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.766515 985 consensus_queue.cc:1035] T 917d5a979ab44260838db6484408c453 P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.779506 985 consensus_queue.cc:1035] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.784008 913 consensus_queue.cc:1035] T fe588e82efae459abbe85693d4f77c14 P 1e63aa3297ad4acd8856e6aa28008957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.797776 581 raft_consensus.cc:1261] T 4b3f550660c441f381045fd89143ec14 P 1e63aa3297ad4acd8856e6aa28008957 [term 1 FOLLOWER]: Refusing update from remote peer 6c6e5fc1b40e4015a97c69fef00a4cff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:10.799279 841 raft_consensus.cc:1261] T 4b3f550660c441f381045fd89143ec14 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Refusing update from remote peer 6c6e5fc1b40e4015a97c69fef00a4cff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:10.799755 954 consensus_queue.cc:1035] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.800084 954 consensus_queue.cc:1035] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.807237 973 consensus_queue.cc:1035] T 4b3f550660c441f381045fd89143ec14 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.808022 844 raft_consensus.cc:1261] T 2ceab1e690b845fd974f0f7dd563c768 P 8a1a8654e29140f184ff9da4d55aceeb [term 1 FOLLOWER]: Refusing update from remote peer 6c6e5fc1b40e4015a97c69fef00a4cff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:10.809026 924 consensus_queue.cc:1035] T 2ceab1e690b845fd974f0f7dd563c768 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.829294 919 consensus_queue.cc:1035] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.830355 996 mvcc.cc:205] Tried to move back new op lower bound from 7022798236860227584 to 7022798235381719040. Current Snapshot: MvccSnapshot[applied={T|T < 7022798236860227584}]
I20240501 07:59:10.832504 1076 mvcc.cc:205] Tried to move back new op lower bound from 7022798236860227584 to 7022798235381719040. Current Snapshot: MvccSnapshot[applied={T|T < 7022798236860227584}]
I20240501 07:59:10.846576 967 consensus_queue.cc:1035] T f468589b44464314a74d96dd6092603d P 8a1a8654e29140f184ff9da4d55aceeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c6e5fc1b40e4015a97c69fef00a4cff" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.856353 997 mvcc.cc:205] Tried to move back new op lower bound from 7022798236860227584 to 7022798235381719040. Current Snapshot: MvccSnapshot[applied={T|T < 7022798236860227584}]
INSERT report
rows total: 2000
time total: 166.111 ms
time per row: 0.0830556 ms
I20240501 07:59:10.932261 1065 consensus_queue.cc:1035] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e63aa3297ad4acd8856e6aa28008957" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:10.932946 1065 consensus_queue.cc:1035] T 2382ca3cb43a4cff9b4594a9936ac9e9 P 6c6e5fc1b40e4015a97c69fef00a4cff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1a8654e29140f184ff9da4d55aceeb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:11.079056 562 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:11.079737 694 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 07:59:11.080415 824 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:11.175755 562 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:11.175806 694 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 07:59:11.178433 824 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:11.291999 562 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:11.293610 824 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:11.293690 694 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 07:59:11.537119 824 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:11.537230 562 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:11.537932 694 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 07:59:11.636577 562 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:11.636718 824 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:11.637621 694 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 07:59:11.668289 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 501
I20240501 07:59:11.674003 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 632
W20240501 07:59:11.679270 780 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.99.129:36459: connect: Connection refused (error 111)
I20240501 07:59:11.679453 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 763
I20240501 07:59:11.684722 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 410
2024-05-01T07:59:11Z chronyd exiting
[ OK ] TableRangeRebalancingTest.Basic (6409 ms)
[----------] 1 test from TableRangeRebalancingTest (6409 ms total)
[----------] 1 test from RebalanceStartSafetyTest
[ RUN ] RebalanceStartSafetyTest.TooManyIgnoredTservers/1
I20240501 07:59:11.711762 398 test_util.cc:268] Using random seed: -592851443
I20240501 07:59:11.712523 398 ts_itest-base.cc:112] Starting cluster with:
I20240501 07:59:11.712564 398 ts_itest-base.cc:113] --------------
I20240501 07:59:11.712579 398 ts_itest-base.cc:114] 5 tablet servers
I20240501 07:59:11.712590 398 ts_itest-base.cc:115] 3 replicas per TS
I20240501 07:59:11.712603 398 ts_itest-base.cc:116] --------------
2024-05-01T07:59:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T07:59:11Z Disabled control of system clock
I20240501 07:59:11.719965 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:44749
--webserver_interface=127.0.99.190
--webserver_port=0
--builtin_ntp_servers=127.0.99.148:41547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.99.190:44749
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240501 07:59:11.824362 1248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:11.824561 1248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:11.824604 1248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:11.827526 1248 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:11.827584 1248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 07:59:11.827633 1248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:11.827666 1248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 07:59:11.827703 1248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 07:59:11.831230 1248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.0.99.190:44749
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:44749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.99.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.1248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:11.832075 1248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:11.833757 1248 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240501 07:59:11.841332 1248 server_base.cc:1034] running on GCE node
W20240501 07:59:11.841769 1254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:11.842262 1253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:11.842722 1256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:11.843175 1248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:11.844102 1248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:11.845240 1248 hybrid_clock.cc:648] HybridClock initialized: now 1714550351845226 us; error 31 us; skew 500 ppm
I20240501 07:59:11.846891 1248 webserver.cc:460] Webserver started at http://127.0.99.190:37641/ using document root <none> and password file <none>
I20240501 07:59:11.847389 1248 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:11.847450 1248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:11.847630 1248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:11.849563 1248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "28c7b6f80fca483185202da0ac60aa23"
format_stamp: "Formatted at 2024-05-01 07:59:11 on dist-test-slave-f81v"
I20240501 07:59:11.849999 1248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "28c7b6f80fca483185202da0ac60aa23"
format_stamp: "Formatted at 2024-05-01 07:59:11 on dist-test-slave-f81v"
I20240501 07:59:11.853080 1248 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240501 07:59:11.855011 1262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:11.855955 1248 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 07:59:11.856046 1248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
uuid: "28c7b6f80fca483185202da0ac60aa23"
format_stamp: "Formatted at 2024-05-01 07:59:11 on dist-test-slave-f81v"
I20240501 07:59:11.856112 1248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:11.875733 1248 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:11.876379 1248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:11.876509 1248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:11.882308 1314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:44749 every 8 connection(s)
I20240501 07:59:11.882318 1248 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:44749
I20240501 07:59:11.883216 1248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 07:59:11.884379 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 1248
I20240501 07:59:11.884485 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 07:59:11.886505 1315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:11.891745 1315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23: Bootstrap starting.
I20240501 07:59:11.893774 1315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:11.894412 1315 log.cc:824] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:11.895888 1315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23: No bootstrap required, opened a new log
I20240501 07:59:11.898176 1315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c7b6f80fca483185202da0ac60aa23" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 44749 } }
I20240501 07:59:11.898347 1315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:11.898398 1315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c7b6f80fca483185202da0ac60aa23, State: Initialized, Role: FOLLOWER
I20240501 07:59:11.898779 1315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c7b6f80fca483185202da0ac60aa23" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 44749 } }
I20240501 07:59:11.898897 1315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 07:59:11.898953 1315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 07:59:11.899044 1315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:11.930887 1315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c7b6f80fca483185202da0ac60aa23" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 44749 } }
I20240501 07:59:11.931315 1315 leader_election.cc:302] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28c7b6f80fca483185202da0ac60aa23; no voters:
I20240501 07:59:11.931638 1315 leader_election.cc:288] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 07:59:11.931756 1320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:11.931999 1320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [term 1 LEADER]: Becoming Leader. State: Replica: 28c7b6f80fca483185202da0ac60aa23, State: Running, Role: LEADER
I20240501 07:59:11.932334 1320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c7b6f80fca483185202da0ac60aa23" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 44749 } }
I20240501 07:59:11.932871 1315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 07:59:11.933902 1322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28c7b6f80fca483185202da0ac60aa23. Latest consensus state: current_term: 1 leader_uuid: "28c7b6f80fca483185202da0ac60aa23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c7b6f80fca483185202da0ac60aa23" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 44749 } } }
I20240501 07:59:11.934023 1322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [sys.catalog]: This master's current role is: LEADER
I20240501 07:59:11.934303 1321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28c7b6f80fca483185202da0ac60aa23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c7b6f80fca483185202da0ac60aa23" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 44749 } } }
I20240501 07:59:11.934389 1321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23 [sys.catalog]: This master's current role is: LEADER
I20240501 07:59:11.934774 1329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 07:59:11.937835 1329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 07:59:11.942451 1329 catalog_manager.cc:1347] Generated new cluster ID: 2cd19e31581647ce9878a0695dd9a8d9
I20240501 07:59:11.942518 1329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 07:59:11.971119 1329 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 07:59:11.972085 1329 catalog_manager.cc:1504] Loading token signing keys...
I20240501 07:59:11.980453 1329 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 28c7b6f80fca483185202da0ac60aa23: Generated new TSK 0
I20240501 07:59:11.981122 1329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 07:59:11.994180 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.129:0
--local_ip_for_outbound_sockets=127.0.99.129
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44749
--builtin_ntp_servers=127.0.99.148:41547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 07:59:12.097695 1339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:12.097903 1339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:12.097962 1339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:12.100878 1339 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:12.100944 1339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:12.101034 1339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.129
I20240501 07:59:12.105635 1339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44749
--never_fsync=true
--heap_profile_path=/tmp/kudu.1339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.129
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:12.106617 1339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:12.107674 1339 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:12.115748 1344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:12.116225 1347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:12.116747 1345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:12.117249 1339 server_base.cc:1034] running on GCE node
I20240501 07:59:12.117587 1339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:12.118114 1339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:12.119268 1339 hybrid_clock.cc:648] HybridClock initialized: now 1714550352119243 us; error 37 us; skew 500 ppm
I20240501 07:59:12.121184 1339 webserver.cc:460] Webserver started at http://127.0.99.129:43507/ using document root <none> and password file <none>
I20240501 07:59:12.121726 1339 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:12.121781 1339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:12.121963 1339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:12.123556 1339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "4d2c04b6062742a5ad5df7d2d2805125"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.124027 1339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "4d2c04b6062742a5ad5df7d2d2805125"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.127061 1339 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240501 07:59:12.129142 1353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.130060 1339 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:12.130157 1339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "4d2c04b6062742a5ad5df7d2d2805125"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.130234 1339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:12.145443 1339 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:12.145952 1339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:12.146090 1339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:12.146564 1339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:12.147368 1339 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:12.147413 1339 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.147452 1339 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:12.147480 1339 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.155182 1339 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:42705
I20240501 07:59:12.155211 1466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:42705 every 8 connection(s)
I20240501 07:59:12.156067 1339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 07:59:12.159242 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 1339
I20240501 07:59:12.159335 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 07:59:12.161492 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:0
--local_ip_for_outbound_sockets=127.0.99.130
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44749
--builtin_ntp_servers=127.0.99.148:41547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 07:59:12.164777 1467 heartbeater.cc:347] Connected to a master server at 127.0.99.190:44749
I20240501 07:59:12.164984 1467 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:12.165454 1467 heartbeater.cc:511] Master 127.0.99.190:44749 requested a full tablet report, sending...
I20240501 07:59:12.166749 1278 ts_manager.cc:194] Registered new tserver with Master: 4d2c04b6062742a5ad5df7d2d2805125 (127.0.99.129:42705)
I20240501 07:59:12.167872 1278 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.129:47751
W20240501 07:59:12.265786 1470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:12.265985 1470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:12.266044 1470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:12.269290 1470 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:12.269353 1470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:12.269440 1470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240501 07:59:12.272877 1470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44749
--never_fsync=true
--heap_profile_path=/tmp/kudu.1470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:12.273870 1470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:12.274919 1470 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:12.282886 1475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:12.283390 1476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:12.283916 1478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:12.284382 1470 server_base.cc:1034] running on GCE node
I20240501 07:59:12.284778 1470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:12.285403 1470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:12.286579 1470 hybrid_clock.cc:648] HybridClock initialized: now 1714550352286557 us; error 33 us; skew 500 ppm
I20240501 07:59:12.288744 1470 webserver.cc:460] Webserver started at http://127.0.99.130:46637/ using document root <none> and password file <none>
I20240501 07:59:12.289346 1470 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:12.289402 1470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:12.289549 1470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:12.291090 1470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "0a1a217f474d48f2a01bef118ea8e3e8"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.291502 1470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "0a1a217f474d48f2a01bef118ea8e3e8"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.294533 1470 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240501 07:59:12.296561 1484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.297493 1470 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240501 07:59:12.297598 1470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "0a1a217f474d48f2a01bef118ea8e3e8"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.297678 1470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:12.324681 1470 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:12.325232 1470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:12.325378 1470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:12.325870 1470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:12.326720 1470 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:12.326764 1470 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.326805 1470 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:12.326833 1470 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.335047 1470 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:34087
I20240501 07:59:12.335093 1597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:34087 every 8 connection(s)
I20240501 07:59:12.335986 1470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 07:59:12.336254 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 1470
I20240501 07:59:12.336369 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 07:59:12.338506 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:0
--local_ip_for_outbound_sockets=127.0.99.131
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44749
--builtin_ntp_servers=127.0.99.148:41547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 07:59:12.345609 1598 heartbeater.cc:347] Connected to a master server at 127.0.99.190:44749
I20240501 07:59:12.345834 1598 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:12.346458 1598 heartbeater.cc:511] Master 127.0.99.190:44749 requested a full tablet report, sending...
I20240501 07:59:12.347461 1278 ts_manager.cc:194] Registered new tserver with Master: 0a1a217f474d48f2a01bef118ea8e3e8 (127.0.99.130:34087)
I20240501 07:59:12.348069 1278 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:42883
W20240501 07:59:12.443882 1601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:12.444103 1601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:12.444155 1601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:12.447050 1601 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:12.447103 1601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:12.447196 1601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240501 07:59:12.450726 1601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44749
--never_fsync=true
--heap_profile_path=/tmp/kudu.1601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:12.451488 1601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:12.452539 1601 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:12.460676 1606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:12.461162 1607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:12.461663 1609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:12.462172 1601 server_base.cc:1034] running on GCE node
I20240501 07:59:12.462486 1601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:12.462987 1601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:12.464135 1601 hybrid_clock.cc:648] HybridClock initialized: now 1714550352464118 us; error 30 us; skew 500 ppm
I20240501 07:59:12.465829 1601 webserver.cc:460] Webserver started at http://127.0.99.131:45445/ using document root <none> and password file <none>
I20240501 07:59:12.466353 1601 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:12.466410 1601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:12.466568 1601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:12.468178 1601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "d0923714aa0a41fbbf9a51d95e0b8637"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.468608 1601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "d0923714aa0a41fbbf9a51d95e0b8637"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.471606 1601 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20240501 07:59:12.473611 1615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.474553 1601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:12.474730 1601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "d0923714aa0a41fbbf9a51d95e0b8637"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.474807 1601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:12.488094 1601 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:12.488619 1601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:12.488749 1601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:12.489233 1601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:12.490036 1601 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:12.490082 1601 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.490134 1601 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:12.490161 1601 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.497620 1601 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:36473
I20240501 07:59:12.497660 1728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:36473 every 8 connection(s)
I20240501 07:59:12.498454 1601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 07:59:12.503619 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 1601
I20240501 07:59:12.503713 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 07:59:12.505697 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:0
--local_ip_for_outbound_sockets=127.0.99.132
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44749
--builtin_ntp_servers=127.0.99.148:41547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 07:59:12.506552 1729 heartbeater.cc:347] Connected to a master server at 127.0.99.190:44749
I20240501 07:59:12.506736 1729 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:12.507292 1729 heartbeater.cc:511] Master 127.0.99.190:44749 requested a full tablet report, sending...
I20240501 07:59:12.508258 1278 ts_manager.cc:194] Registered new tserver with Master: d0923714aa0a41fbbf9a51d95e0b8637 (127.0.99.131:36473)
I20240501 07:59:12.508798 1278 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:36511
W20240501 07:59:12.605489 1732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:12.605695 1732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:12.605751 1732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:12.608687 1732 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:12.608747 1732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:12.608826 1732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240501 07:59:12.612272 1732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44749
--never_fsync=true
--heap_profile_path=/tmp/kudu.1732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:12.613128 1732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:12.614209 1732 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:12.622131 1737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:12.622624 1738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:12.623092 1740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:12.623608 1732 server_base.cc:1034] running on GCE node
I20240501 07:59:12.623956 1732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:12.624444 1732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:12.625577 1732 hybrid_clock.cc:648] HybridClock initialized: now 1714550352625561 us; error 30 us; skew 500 ppm
I20240501 07:59:12.627311 1732 webserver.cc:460] Webserver started at http://127.0.99.132:40543/ using document root <none> and password file <none>
I20240501 07:59:12.627815 1732 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:12.627867 1732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:12.628072 1732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:12.629683 1732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "3777b7f6493d43b081fc1ed8041f35d0"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.630141 1732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "3777b7f6493d43b081fc1ed8041f35d0"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.633147 1732 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 07:59:12.635164 1746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.636178 1732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 07:59:12.636267 1732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "3777b7f6493d43b081fc1ed8041f35d0"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.636350 1732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:12.652823 1732 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:12.653347 1732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:12.653486 1732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:12.653954 1732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:12.654749 1732 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:12.654798 1732 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.654851 1732 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:12.654879 1732 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.662894 1732 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:36351
I20240501 07:59:12.662910 1859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:36351 every 8 connection(s)
I20240501 07:59:12.663800 1732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 07:59:12.670513 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 1732
I20240501 07:59:12.670601 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 07:59:12.672070 1860 heartbeater.cc:347] Connected to a master server at 127.0.99.190:44749
I20240501 07:59:12.672292 1860 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:12.672556 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:0
--local_ip_for_outbound_sockets=127.0.99.133
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44749
--builtin_ntp_servers=127.0.99.148:41547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 07:59:12.672799 1860 heartbeater.cc:511] Master 127.0.99.190:44749 requested a full tablet report, sending...
I20240501 07:59:12.673722 1278 ts_manager.cc:194] Registered new tserver with Master: 3777b7f6493d43b081fc1ed8041f35d0 (127.0.99.132:36351)
I20240501 07:59:12.674378 1278 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:39665
W20240501 07:59:12.775266 1863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:12.775476 1863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:12.775535 1863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:12.778555 1863 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:12.778620 1863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:12.778698 1863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240501 07:59:12.782203 1863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:44749
--never_fsync=true
--heap_profile_path=/tmp/kudu.1863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:12.783038 1863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:12.784137 1863 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:12.792285 1868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:12.793285 1869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:12.793309 1871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:12.793780 1863 server_base.cc:1034] running on GCE node
I20240501 07:59:12.794221 1863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:12.794831 1863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:12.795982 1863 hybrid_clock.cc:648] HybridClock initialized: now 1714550352795957 us; error 37 us; skew 500 ppm
I20240501 07:59:12.798159 1863 webserver.cc:460] Webserver started at http://127.0.99.133:43617/ using document root <none> and password file <none>
I20240501 07:59:12.798652 1863 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:12.798699 1863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:12.798846 1863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:12.800442 1863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "5e1174fac6274ce0badcc40d89c6f259"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.800873 1863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "5e1174fac6274ce0badcc40d89c6f259"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.803973 1863 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 07:59:12.805970 1877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.806870 1863 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:12.806969 1863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "5e1174fac6274ce0badcc40d89c6f259"
format_stamp: "Formatted at 2024-05-01 07:59:12 on dist-test-slave-f81v"
I20240501 07:59:12.807047 1863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:12.827533 1863 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:12.828126 1863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:12.828267 1863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:12.828732 1863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:12.830003 1863 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:12.830049 1863 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.830098 1863 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:12.830125 1863 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:12.838161 1863 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:36401
I20240501 07:59:12.838183 1990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:36401 every 8 connection(s)
I20240501 07:59:12.839100 1863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 07:59:12.847009 1991 heartbeater.cc:347] Connected to a master server at 127.0.99.190:44749
I20240501 07:59:12.847213 1991 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:12.847656 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 1863
I20240501 07:59:12.847707 1991 heartbeater.cc:511] Master 127.0.99.190:44749 requested a full tablet report, sending...
I20240501 07:59:12.847759 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240501 07:59:12.848652 1278 ts_manager.cc:194] Registered new tserver with Master: 5e1174fac6274ce0badcc40d89c6f259 (127.0.99.133:36401)
I20240501 07:59:12.849208 1278 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:54767
I20240501 07:59:12.849784 398 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240501 07:59:12.857579 1279 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39566:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240501 07:59:12.873945 1663 tablet_service.cc:1467] Processing CreateTablet for tablet 1223cb283cda4adc8fe31bcdc5d63c99 (DEFAULT_TABLE table=TestTable [id=90cec7360c5a476a9ea18631c2e63e28]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:12.874035 1401 tablet_service.cc:1467] Processing CreateTablet for tablet 1223cb283cda4adc8fe31bcdc5d63c99 (DEFAULT_TABLE table=TestTable [id=90cec7360c5a476a9ea18631c2e63e28]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:12.874020 1532 tablet_service.cc:1467] Processing CreateTablet for tablet 1223cb283cda4adc8fe31bcdc5d63c99 (DEFAULT_TABLE table=TestTable [id=90cec7360c5a476a9ea18631c2e63e28]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:12.874869 1532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1223cb283cda4adc8fe31bcdc5d63c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:12.874877 1663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1223cb283cda4adc8fe31bcdc5d63c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:12.874877 1401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1223cb283cda4adc8fe31bcdc5d63c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:12.879801 2008 tablet_bootstrap.cc:492] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8: Bootstrap starting.
I20240501 07:59:12.880775 2010 tablet_bootstrap.cc:492] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637: Bootstrap starting.
I20240501 07:59:12.881918 2008 tablet_bootstrap.cc:654] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:12.882371 2010 tablet_bootstrap.cc:654] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:12.882373 2009 tablet_bootstrap.cc:492] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125: Bootstrap starting.
I20240501 07:59:12.883105 2010 log.cc:824] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:12.883376 2008 log.cc:824] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:12.884168 2009 tablet_bootstrap.cc:654] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:12.884840 2010 tablet_bootstrap.cc:492] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637: No bootstrap required, opened a new log
I20240501 07:59:12.884833 2009 log.cc:824] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:12.885015 2010 ts_tablet_manager.cc:1367] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 07:59:12.885335 2008 tablet_bootstrap.cc:492] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8: No bootstrap required, opened a new log
I20240501 07:59:12.885535 2008 ts_tablet_manager.cc:1367] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240501 07:59:12.886257 2009 tablet_bootstrap.cc:492] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125: No bootstrap required, opened a new log
I20240501 07:59:12.886420 2009 ts_tablet_manager.cc:1367] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 07:59:12.889040 2008 raft_consensus.cc:348] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0923714aa0a41fbbf9a51d95e0b8637" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36473 } } peers { permanent_uuid: "0a1a217f474d48f2a01bef118ea8e3e8" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 34087 } } peers { permanent_uuid: "4d2c04b6062742a5ad5df7d2d2805125" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42705 } }
I20240501 07:59:12.889330 2008 raft_consensus.cc:374] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:12.889410 2008 raft_consensus.cc:729] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1a217f474d48f2a01bef118ea8e3e8, State: Initialized, Role: FOLLOWER
I20240501 07:59:12.889348 2010 raft_consensus.cc:348] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0923714aa0a41fbbf9a51d95e0b8637" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36473 } } peers { permanent_uuid: "0a1a217f474d48f2a01bef118ea8e3e8" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 34087 } } peers { permanent_uuid: "4d2c04b6062742a5ad5df7d2d2805125" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42705 } }
I20240501 07:59:12.889534 2010 raft_consensus.cc:374] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:12.889591 2010 raft_consensus.cc:729] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0923714aa0a41fbbf9a51d95e0b8637, State: Initialized, Role: FOLLOWER
I20240501 07:59:12.889820 2009 raft_consensus.cc:348] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0923714aa0a41fbbf9a51d95e0b8637" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36473 } } peers { permanent_uuid: "0a1a217f474d48f2a01bef118ea8e3e8" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 34087 } } peers { permanent_uuid: "4d2c04b6062742a5ad5df7d2d2805125" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42705 } }
I20240501 07:59:12.890010 2009 raft_consensus.cc:374] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:12.890064 2009 raft_consensus.cc:729] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d2c04b6062742a5ad5df7d2d2805125, State: Initialized, Role: FOLLOWER
I20240501 07:59:12.890002 2010 consensus_queue.cc:260] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0923714aa0a41fbbf9a51d95e0b8637" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36473 } } peers { permanent_uuid: "0a1a217f474d48f2a01bef118ea8e3e8" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 34087 } } peers { permanent_uuid: "4d2c04b6062742a5ad5df7d2d2805125" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42705 } }
I20240501 07:59:12.890082 2008 consensus_queue.cc:260] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0923714aa0a41fbbf9a51d95e0b8637" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36473 } } peers { permanent_uuid: "0a1a217f474d48f2a01bef118ea8e3e8" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 34087 } } peers { permanent_uuid: "4d2c04b6062742a5ad5df7d2d2805125" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42705 } }
I20240501 07:59:12.890458 2009 consensus_queue.cc:260] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0923714aa0a41fbbf9a51d95e0b8637" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36473 } } peers { permanent_uuid: "0a1a217f474d48f2a01bef118ea8e3e8" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 34087 } } peers { permanent_uuid: "4d2c04b6062742a5ad5df7d2d2805125" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42705 } }
I20240501 07:59:12.890801 2010 ts_tablet_manager.cc:1398] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 07:59:12.891140 1729 heartbeater.cc:503] Master 127.0.99.190:44749 was elected leader, sending a full tablet report...
I20240501 07:59:12.891158 2008 ts_tablet_manager.cc:1398] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20240501 07:59:12.891466 1598 heartbeater.cc:503] Master 127.0.99.190:44749 was elected leader, sending a full tablet report...
I20240501 07:59:12.891702 2009 ts_tablet_manager.cc:1398] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240501 07:59:12.892274 1467 heartbeater.cc:503] Master 127.0.99.190:44749 was elected leader, sending a full tablet report...
W20240501 07:59:12.907168 1468 tablet.cc:2369] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 07:59:12.999393 1730 tablet.cc:2369] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 07:59:13.068092 2016 raft_consensus.cc:482] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:13.068297 2016 raft_consensus.cc:504] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0923714aa0a41fbbf9a51d95e0b8637" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36473 } } peers { permanent_uuid: "0a1a217f474d48f2a01bef118ea8e3e8" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 34087 } } peers { permanent_uuid: "4d2c04b6062742a5ad5df7d2d2805125" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42705 } }
I20240501 07:59:13.069419 2016 leader_election.cc:288] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0923714aa0a41fbbf9a51d95e0b8637 (127.0.99.131:36473), 0a1a217f474d48f2a01bef118ea8e3e8 (127.0.99.130:34087)
I20240501 07:59:13.073256 1683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1223cb283cda4adc8fe31bcdc5d63c99" candidate_uuid: "4d2c04b6062742a5ad5df7d2d2805125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0923714aa0a41fbbf9a51d95e0b8637" is_pre_election: true
I20240501 07:59:13.073262 1552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1223cb283cda4adc8fe31bcdc5d63c99" candidate_uuid: "4d2c04b6062742a5ad5df7d2d2805125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1a217f474d48f2a01bef118ea8e3e8" is_pre_election: true
I20240501 07:59:13.073562 1683 raft_consensus.cc:2454] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d2c04b6062742a5ad5df7d2d2805125 in term 0.
I20240501 07:59:13.073562 1552 raft_consensus.cc:2454] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d2c04b6062742a5ad5df7d2d2805125 in term 0.
I20240501 07:59:13.074018 1356 leader_election.cc:302] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1a217f474d48f2a01bef118ea8e3e8, 4d2c04b6062742a5ad5df7d2d2805125; no voters:
I20240501 07:59:13.074280 2016 raft_consensus.cc:2781] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:13.074352 2016 raft_consensus.cc:482] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:13.074399 2016 raft_consensus.cc:3037] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [term 0 FOLLOWER]: Advancing to term 1
W20240501 07:59:13.087692 1599 tablet.cc:2369] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 07:59:13.153475 2016 raft_consensus.cc:504] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0923714aa0a41fbbf9a51d95e0b8637" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36473 } } peers { permanent_uuid: "0a1a217f474d48f2a01bef118ea8e3e8" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 34087 } } peers { permanent_uuid: "4d2c04b6062742a5ad5df7d2d2805125" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42705 } }
I20240501 07:59:13.153911 2016 leader_election.cc:288] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [CANDIDATE]: Term 1 election: Requested vote from peers d0923714aa0a41fbbf9a51d95e0b8637 (127.0.99.131:36473), 0a1a217f474d48f2a01bef118ea8e3e8 (127.0.99.130:34087)
I20240501 07:59:13.154318 1683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1223cb283cda4adc8fe31bcdc5d63c99" candidate_uuid: "4d2c04b6062742a5ad5df7d2d2805125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0923714aa0a41fbbf9a51d95e0b8637"
I20240501 07:59:13.154350 1552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1223cb283cda4adc8fe31bcdc5d63c99" candidate_uuid: "4d2c04b6062742a5ad5df7d2d2805125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1a217f474d48f2a01bef118ea8e3e8"
I20240501 07:59:13.154470 1683 raft_consensus.cc:3037] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:13.154470 1552 raft_consensus.cc:3037] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:13.155530 1552 raft_consensus.cc:2454] T 1223cb283cda4adc8fe31bcdc5d63c99 P 0a1a217f474d48f2a01bef118ea8e3e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d2c04b6062742a5ad5df7d2d2805125 in term 1.
I20240501 07:59:13.155560 1683 raft_consensus.cc:2454] T 1223cb283cda4adc8fe31bcdc5d63c99 P d0923714aa0a41fbbf9a51d95e0b8637 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d2c04b6062742a5ad5df7d2d2805125 in term 1.
I20240501 07:59:13.155875 1356 leader_election.cc:302] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1a217f474d48f2a01bef118ea8e3e8, 4d2c04b6062742a5ad5df7d2d2805125; no voters:
I20240501 07:59:13.156466 2016 raft_consensus.cc:2781] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:13.156677 2016 raft_consensus.cc:686] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [term 1 LEADER]: Becoming Leader. State: Replica: 4d2c04b6062742a5ad5df7d2d2805125, State: Running, Role: LEADER
I20240501 07:59:13.156981 2016 consensus_queue.cc:237] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0923714aa0a41fbbf9a51d95e0b8637" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36473 } } peers { permanent_uuid: "0a1a217f474d48f2a01bef118ea8e3e8" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 34087 } } peers { permanent_uuid: "4d2c04b6062742a5ad5df7d2d2805125" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42705 } }
I20240501 07:59:13.160099 1279 catalog_manager.cc:5484] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d2c04b6062742a5ad5df7d2d2805125 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "4d2c04b6062742a5ad5df7d2d2805125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0923714aa0a41fbbf9a51d95e0b8637" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1a217f474d48f2a01bef118ea8e3e8" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 34087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d2c04b6062742a5ad5df7d2d2805125" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42705 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:13.227262 398 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240501 07:59:13.229162 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 4d2c04b6062742a5ad5df7d2d2805125 to finish bootstrapping
I20240501 07:59:13.234053 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 0a1a217f474d48f2a01bef118ea8e3e8 to finish bootstrapping
I20240501 07:59:13.238278 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver d0923714aa0a41fbbf9a51d95e0b8637 to finish bootstrapping
I20240501 07:59:13.242595 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 3777b7f6493d43b081fc1ed8041f35d0 to finish bootstrapping
I20240501 07:59:13.247692 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 5e1174fac6274ce0badcc40d89c6f259 to finish bootstrapping
I20240501 07:59:13.429433 1925 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.430291 1663 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.430291 1532 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.431392 1794 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.432636 1401 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:13.520850 1794 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.521690 1532 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.521881 1401 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:13.523257 1663 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.528112 1925 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.561843 2016 consensus_queue.cc:1035] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0923714aa0a41fbbf9a51d95e0b8637" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:13.566891 2016 consensus_queue.cc:1035] T 1223cb283cda4adc8fe31bcdc5d63c99 P 4d2c04b6062742a5ad5df7d2d2805125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1a217f474d48f2a01bef118ea8e3e8" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 34087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:13.626621 1532 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.627445 1794 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.627444 1401 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:13.636641 1663 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.640967 1925 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.676108 1860 heartbeater.cc:503] Master 127.0.99.190:44749 was elected leader, sending a full tablet report...
I20240501 07:59:13.846182 1794 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.847671 1532 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.848774 1401 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:13.851159 1991 heartbeater.cc:503] Master 127.0.99.190:44749 was elected leader, sending a full tablet report...
I20240501 07:59:13.856717 1925 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.873679 1663 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.945240 1663 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.945242 1794 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.946274 1925 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.960757 1532 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:13.961227 1401 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:14.047839 1401 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:14.047978 1532 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:14.048135 1925 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:14.048532 1794 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:14.061300 1663 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:14.087384 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 1339
I20240501 07:59:14.092523 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 1470
I20240501 07:59:14.097057 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 1601
I20240501 07:59:14.101387 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 1732
I20240501 07:59:14.105577 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 1863
I20240501 07:59:14.110067 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 1248
2024-05-01T07:59:14Z chronyd exiting
[ OK ] RebalanceStartSafetyTest.TooManyIgnoredTservers/1 (2418 ms)
[----------] 1 test from RebalanceStartSafetyTest (2418 ms total)
[----------] 1 test from RebalanceParamTest
[ RUN ] RebalanceParamTest.Rebalance/7
I20240501 07:59:14.129873 398 test_util.cc:268] Using random seed: -590433332
I20240501 07:59:14.130527 398 ts_itest-base.cc:112] Starting cluster with:
I20240501 07:59:14.130570 398 ts_itest-base.cc:113] --------------
I20240501 07:59:14.130580 398 ts_itest-base.cc:114] 7 tablet servers
I20240501 07:59:14.130589 398 ts_itest-base.cc:115] 5 replicas per TS
I20240501 07:59:14.130604 398 ts_itest-base.cc:116] --------------
2024-05-01T07:59:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T07:59:14Z Disabled control of system clock
I20240501 07:59:14.137195 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:35233
--webserver_interface=127.0.99.190
--webserver_port=0
--builtin_ntp_servers=127.0.99.148:36633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.99.190:35233
--allow_unsafe_replication_factor
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--tserver_unresponsive_timeout_ms=3000 with env {}
W20240501 07:59:14.236205 2236 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240501 07:59:14.236420 2236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:14.236450 2236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:14.236510 2236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:14.239457 2236 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:14.239521 2236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 07:59:14.239547 2236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:14.239578 2236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 07:59:14.239607 2236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 07:59:14.243121 2236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:36633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.0.99.190:35233
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:35233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.99.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.2236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:14.243919 2236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:14.245050 2236 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:14.252081 2241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:14.252540 2244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:14.253012 2242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:14.253551 2236 server_base.cc:1034] running on GCE node
I20240501 07:59:14.253983 2236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:14.254815 2236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:14.255971 2236 hybrid_clock.cc:648] HybridClock initialized: now 1714550354255963 us; error 31 us; skew 500 ppm
I20240501 07:59:14.257678 2236 webserver.cc:460] Webserver started at http://127.0.99.190:45419/ using document root <none> and password file <none>
I20240501 07:59:14.258191 2236 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:14.258241 2236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:14.258405 2236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:14.259989 2236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "9c220cab46b743088853d0027749db0d"
format_stamp: "Formatted at 2024-05-01 07:59:14 on dist-test-slave-f81v"
I20240501 07:59:14.260459 2236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "9c220cab46b743088853d0027749db0d"
format_stamp: "Formatted at 2024-05-01 07:59:14 on dist-test-slave-f81v"
I20240501 07:59:14.263507 2236 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20240501 07:59:14.265516 2250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:14.266454 2236 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:14.266548 2236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
uuid: "9c220cab46b743088853d0027749db0d"
format_stamp: "Formatted at 2024-05-01 07:59:14 on dist-test-slave-f81v"
I20240501 07:59:14.266631 2236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:14.285761 2236 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:14.286368 2236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:14.286512 2236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:14.293115 2236 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:35233
I20240501 07:59:14.293114 2302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:35233 every 8 connection(s)
I20240501 07:59:14.294051 2236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 07:59:14.296427 2303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:14.301045 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 2236
I20240501 07:59:14.301136 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 07:59:14.301190 2303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d: Bootstrap starting.
I20240501 07:59:14.303406 2303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:14.304059 2303 log.cc:824] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:14.305505 2303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d: No bootstrap required, opened a new log
I20240501 07:59:14.308012 2303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c220cab46b743088853d0027749db0d" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 35233 } }
I20240501 07:59:14.308214 2303 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:14.308279 2303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c220cab46b743088853d0027749db0d, State: Initialized, Role: FOLLOWER
I20240501 07:59:14.308701 2303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c220cab46b743088853d0027749db0d" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 35233 } }
I20240501 07:59:14.308815 2303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 07:59:14.308871 2303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 07:59:14.308976 2303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:14.310731 2303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c220cab46b743088853d0027749db0d" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 35233 } }
I20240501 07:59:14.311061 2303 leader_election.cc:302] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c220cab46b743088853d0027749db0d; no voters:
I20240501 07:59:14.311317 2303 leader_election.cc:288] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 07:59:14.311407 2308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:14.311889 2308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [term 1 LEADER]: Becoming Leader. State: Replica: 9c220cab46b743088853d0027749db0d, State: Running, Role: LEADER
I20240501 07:59:14.312348 2303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [sys.catalog]: configured and running, proceeding with master startup.
I20240501 07:59:14.312461 2308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c220cab46b743088853d0027749db0d" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 35233 } }
I20240501 07:59:14.313448 2309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c220cab46b743088853d0027749db0d. Latest consensus state: current_term: 1 leader_uuid: "9c220cab46b743088853d0027749db0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c220cab46b743088853d0027749db0d" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 35233 } } }
I20240501 07:59:14.313553 2309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [sys.catalog]: This master's current role is: LEADER
I20240501 07:59:14.313715 2310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c220cab46b743088853d0027749db0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c220cab46b743088853d0027749db0d" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 35233 } } }
I20240501 07:59:14.313810 2310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d [sys.catalog]: This master's current role is: LEADER
I20240501 07:59:14.313864 2317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 07:59:14.316745 2317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 07:59:14.321359 2317 catalog_manager.cc:1347] Generated new cluster ID: db282503c35a414a9460735b9c454c4a
I20240501 07:59:14.321425 2317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 07:59:14.370496 2317 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 07:59:14.371471 2317 catalog_manager.cc:1504] Loading token signing keys...
I20240501 07:59:14.377189 2317 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 9c220cab46b743088853d0027749db0d: Generated new TSK 0
I20240501 07:59:14.377833 2317 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 07:59:14.385149 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.129:0
--local_ip_for_outbound_sockets=127.0.99.129
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--builtin_ntp_servers=127.0.99.148:36633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 07:59:14.501684 2327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:14.501871 2327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:14.501917 2327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:14.504891 2327 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:14.504947 2327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:14.505040 2327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.129
I20240501 07:59:14.508510 2327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:36633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--never_fsync=true
--heap_profile_path=/tmp/kudu.2327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.129
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:14.509277 2327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:14.510301 2327 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:14.518299 2333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:14.518779 2335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:14.519457 2332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:14.519999 2327 server_base.cc:1034] running on GCE node
I20240501 07:59:14.520334 2327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:14.520848 2327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:14.522007 2327 hybrid_clock.cc:648] HybridClock initialized: now 1714550354521984 us; error 37 us; skew 500 ppm
I20240501 07:59:14.523756 2327 webserver.cc:460] Webserver started at http://127.0.99.129:43685/ using document root <none> and password file <none>
I20240501 07:59:14.524300 2327 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:14.524355 2327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:14.524523 2327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:14.526041 2327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "85377a976c05408fbb384237a0edb691"
format_stamp: "Formatted at 2024-05-01 07:59:14 on dist-test-slave-f81v"
I20240501 07:59:14.526471 2327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "85377a976c05408fbb384237a0edb691"
format_stamp: "Formatted at 2024-05-01 07:59:14 on dist-test-slave-f81v"
I20240501 07:59:14.529572 2327 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 07:59:14.531607 2341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:14.532562 2327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:14.532658 2327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "85377a976c05408fbb384237a0edb691"
format_stamp: "Formatted at 2024-05-01 07:59:14 on dist-test-slave-f81v"
I20240501 07:59:14.532735 2327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:14.544211 2327 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:14.544744 2327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:14.544883 2327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:14.545348 2327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:14.546157 2327 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:14.546202 2327 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:14.546250 2327 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:14.546278 2327 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:14.553936 2327 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:42933
I20240501 07:59:14.553992 2454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:42933 every 8 connection(s)
I20240501 07:59:14.554773 2327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 07:59:14.560448 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 2327
I20240501 07:59:14.560539 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 07:59:14.562541 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:0
--local_ip_for_outbound_sockets=127.0.99.130
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--builtin_ntp_servers=127.0.99.148:36633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 07:59:14.563036 2455 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35233
I20240501 07:59:14.563233 2455 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:14.563779 2455 heartbeater.cc:511] Master 127.0.99.190:35233 requested a full tablet report, sending...
I20240501 07:59:14.565189 2267 ts_manager.cc:194] Registered new tserver with Master: 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933)
I20240501 07:59:14.566295 2267 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.129:60579
W20240501 07:59:14.666164 2458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:14.666352 2458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:14.666399 2458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:14.669438 2458 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:14.669502 2458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:14.669592 2458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240501 07:59:14.673139 2458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:36633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--never_fsync=true
--heap_profile_path=/tmp/kudu.2458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:14.673950 2458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:14.675001 2458 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:14.682758 2464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:14.683246 2463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:14.683764 2466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:14.684316 2458 server_base.cc:1034] running on GCE node
I20240501 07:59:14.684669 2458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:14.685197 2458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:14.686344 2458 hybrid_clock.cc:648] HybridClock initialized: now 1714550354686323 us; error 37 us; skew 500 ppm
I20240501 07:59:14.688076 2458 webserver.cc:460] Webserver started at http://127.0.99.130:46725/ using document root <none> and password file <none>
I20240501 07:59:14.688598 2458 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:14.688653 2458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:14.688818 2458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:14.690364 2458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "5810fd56ce9244438edcd3c0ea23e72c"
format_stamp: "Formatted at 2024-05-01 07:59:14 on dist-test-slave-f81v"
I20240501 07:59:14.690822 2458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "5810fd56ce9244438edcd3c0ea23e72c"
format_stamp: "Formatted at 2024-05-01 07:59:14 on dist-test-slave-f81v"
I20240501 07:59:14.693864 2458 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 07:59:14.695770 2472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:14.696688 2458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:14.696791 2458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "5810fd56ce9244438edcd3c0ea23e72c"
format_stamp: "Formatted at 2024-05-01 07:59:14 on dist-test-slave-f81v"
I20240501 07:59:14.696864 2458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:14.711251 2458 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:14.711817 2458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:14.711992 2458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:14.712507 2458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:14.713376 2458 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:14.713423 2458 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:14.713472 2458 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:14.713500 2458 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:14.721321 2458 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:32773
I20240501 07:59:14.721345 2585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:32773 every 8 connection(s)
I20240501 07:59:14.722159 2458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 07:59:14.727396 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 2458
I20240501 07:59:14.727492 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 07:59:14.729457 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:0
--local_ip_for_outbound_sockets=127.0.99.131
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--builtin_ntp_servers=127.0.99.148:36633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 07:59:14.731107 2586 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35233
I20240501 07:59:14.731305 2586 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:14.731801 2586 heartbeater.cc:511] Master 127.0.99.190:35233 requested a full tablet report, sending...
I20240501 07:59:14.732755 2267 ts_manager.cc:194] Registered new tserver with Master: 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:14.733462 2267 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:33111
W20240501 07:59:14.832114 2589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:14.832289 2589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:14.832327 2589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:14.835134 2589 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:14.835182 2589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:14.835253 2589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240501 07:59:14.838642 2589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:36633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--never_fsync=true
--heap_profile_path=/tmp/kudu.2589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:14.839408 2589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:14.840575 2589 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:14.848147 2595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:14.848680 2594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:14.849239 2597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:14.849707 2589 server_base.cc:1034] running on GCE node
I20240501 07:59:14.850009 2589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:14.850474 2589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:14.851615 2589 hybrid_clock.cc:648] HybridClock initialized: now 1714550354851592 us; error 34 us; skew 500 ppm
I20240501 07:59:14.853291 2589 webserver.cc:460] Webserver started at http://127.0.99.131:41439/ using document root <none> and password file <none>
I20240501 07:59:14.853767 2589 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:14.853816 2589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:14.853976 2589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:14.855448 2589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
format_stamp: "Formatted at 2024-05-01 07:59:14 on dist-test-slave-f81v"
I20240501 07:59:14.855875 2589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
format_stamp: "Formatted at 2024-05-01 07:59:14 on dist-test-slave-f81v"
I20240501 07:59:14.858783 2589 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:14.860671 2603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:14.861567 2589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:14.861662 2589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
format_stamp: "Formatted at 2024-05-01 07:59:14 on dist-test-slave-f81v"
I20240501 07:59:14.861734 2589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:14.880336 2589 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:14.880889 2589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:14.881023 2589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:14.881494 2589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:14.882309 2589 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:14.882354 2589 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:14.882402 2589 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:14.882428 2589 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:14.890022 2589 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:44519
I20240501 07:59:14.890045 2716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:44519 every 8 connection(s)
I20240501 07:59:14.890839 2589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 07:59:14.895018 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 2589
I20240501 07:59:14.895097 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 07:59:14.896884 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:0
--local_ip_for_outbound_sockets=127.0.99.132
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--builtin_ntp_servers=127.0.99.148:36633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 07:59:14.898603 2717 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35233
I20240501 07:59:14.898797 2717 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:14.899252 2717 heartbeater.cc:511] Master 127.0.99.190:35233 requested a full tablet report, sending...
I20240501 07:59:14.900159 2267 ts_manager.cc:194] Registered new tserver with Master: 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519)
I20240501 07:59:14.900686 2267 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:40721
W20240501 07:59:14.994895 2720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:14.995102 2720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:14.995157 2720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:14.998116 2720 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:14.998181 2720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:14.998261 2720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240501 07:59:15.001765 2720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:36633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--never_fsync=true
--heap_profile_path=/tmp/kudu.2720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:15.002555 2720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:15.003572 2720 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:15.011209 2725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:15.011694 2726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:15.012233 2728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:15.012796 2720 server_base.cc:1034] running on GCE node
I20240501 07:59:15.013101 2720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:15.013585 2720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:15.014729 2720 hybrid_clock.cc:648] HybridClock initialized: now 1714550355014717 us; error 28 us; skew 500 ppm
I20240501 07:59:15.016393 2720 webserver.cc:460] Webserver started at http://127.0.99.132:45911/ using document root <none> and password file <none>
I20240501 07:59:15.016875 2720 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:15.016924 2720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:15.017087 2720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:15.018582 2720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "3bd6898f790b48d593253180b79c5e40"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:15.019008 2720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "3bd6898f790b48d593253180b79c5e40"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:15.021970 2720 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 07:59:15.023789 2734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:15.024693 2720 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240501 07:59:15.024784 2720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "3bd6898f790b48d593253180b79c5e40"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:15.024855 2720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:15.047679 2720 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:15.048295 2720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:15.048429 2720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:15.048897 2720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:15.049705 2720 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:15.049751 2720 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:15.049799 2720 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:15.049826 2720 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:15.057492 2720 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:40829
I20240501 07:59:15.057513 2847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:40829 every 8 connection(s)
I20240501 07:59:15.058297 2720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 07:59:15.061309 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 2720
I20240501 07:59:15.061424 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 07:59:15.063378 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:0
--local_ip_for_outbound_sockets=127.0.99.133
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--builtin_ntp_servers=127.0.99.148:36633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 07:59:15.066259 2848 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35233
I20240501 07:59:15.066478 2848 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:15.067042 2848 heartbeater.cc:511] Master 127.0.99.190:35233 requested a full tablet report, sending...
I20240501 07:59:15.068042 2267 ts_manager.cc:194] Registered new tserver with Master: 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829)
I20240501 07:59:15.068734 2267 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:53989
W20240501 07:59:15.167179 2851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:15.167352 2851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:15.167390 2851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:15.170274 2851 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:15.170326 2851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:15.170387 2851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240501 07:59:15.173732 2851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:36633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--never_fsync=true
--heap_profile_path=/tmp/kudu.2851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:15.174417 2851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:15.175360 2851 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:15.182706 2859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:15.183218 2857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:15.183722 2856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:15.184248 2851 server_base.cc:1034] running on GCE node
I20240501 07:59:15.184552 2851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:15.185012 2851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:15.186154 2851 hybrid_clock.cc:648] HybridClock initialized: now 1714550355186142 us; error 25 us; skew 500 ppm
I20240501 07:59:15.187744 2851 webserver.cc:460] Webserver started at http://127.0.99.133:42489/ using document root <none> and password file <none>
I20240501 07:59:15.188283 2851 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:15.188333 2851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:15.188477 2851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:15.189918 2851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "e2c508eb3dac4193a1eeba799e5d54f2"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:15.190318 2851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "e2c508eb3dac4193a1eeba799e5d54f2"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:15.193220 2851 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 07:59:15.195045 2865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:15.195981 2851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 07:59:15.196072 2851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "e2c508eb3dac4193a1eeba799e5d54f2"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:15.196137 2851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:15.207442 2851 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:15.208135 2851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:15.208297 2851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:15.208832 2851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:15.209820 2851 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:15.209879 2851 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:15.209942 2851 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:15.209972 2851 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:15.218928 2851 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:35591
I20240501 07:59:15.218955 2978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:35591 every 8 connection(s)
I20240501 07:59:15.219722 2851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 07:59:15.227200 2979 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35233
I20240501 07:59:15.227396 2979 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:15.227823 2979 heartbeater.cc:511] Master 127.0.99.190:35233 requested a full tablet report, sending...
I20240501 07:59:15.228605 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 2851
I20240501 07:59:15.228710 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240501 07:59:15.228735 2267 ts_manager.cc:194] Registered new tserver with Master: e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591)
I20240501 07:59:15.229347 2267 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:58171
I20240501 07:59:15.230798 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.134:0
--local_ip_for_outbound_sockets=127.0.99.134
--webserver_interface=127.0.99.134
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--builtin_ntp_servers=127.0.99.148:36633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 07:59:15.330922 2982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:15.331116 2982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:15.331173 2982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:15.334111 2982 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:15.334172 2982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:15.334254 2982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.134
I20240501 07:59:15.337762 2982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:36633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.0.99.134
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--never_fsync=true
--heap_profile_path=/tmp/kudu.2982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.134
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:15.338505 2982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:15.339507 2982 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:15.347155 2990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:15.347604 2988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:15.348111 2987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:15.348634 2982 server_base.cc:1034] running on GCE node
I20240501 07:59:15.348933 2982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:15.349411 2982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:15.350555 2982 hybrid_clock.cc:648] HybridClock initialized: now 1714550355350539 us; error 24 us; skew 500 ppm
I20240501 07:59:15.352519 2982 webserver.cc:460] Webserver started at http://127.0.99.134:40519/ using document root <none> and password file <none>
I20240501 07:59:15.353075 2982 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:15.353132 2982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:15.353284 2982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:15.354784 2982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "1537338fbeea47ee81f7a210cd6df2f0"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:15.355203 2982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "1537338fbeea47ee81f7a210cd6df2f0"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:15.358142 2982 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240501 07:59:15.360083 2996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:15.361016 2982 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:15.361110 2982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "1537338fbeea47ee81f7a210cd6df2f0"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:15.361182 2982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:15.397151 2982 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:15.397747 2982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:15.397893 2982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:15.398434 2982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:15.399327 2982 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:15.399370 2982 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:15.399421 2982 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:15.399444 2982 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:15.407039 2982 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.134:44027
I20240501 07:59:15.407068 3109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.134:44027 every 8 connection(s)
I20240501 07:59:15.407864 2982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240501 07:59:15.415561 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 2982
I20240501 07:59:15.415652 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240501 07:59:15.415721 3110 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35233
I20240501 07:59:15.415956 3110 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:15.416486 3110 heartbeater.cc:511] Master 127.0.99.190:35233 requested a full tablet report, sending...
I20240501 07:59:15.417341 2266 ts_manager.cc:194] Registered new tserver with Master: 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027)
I20240501 07:59:15.417600 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.135:0
--local_ip_for_outbound_sockets=127.0.99.135
--webserver_interface=127.0.99.135
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--builtin_ntp_servers=127.0.99.148:36633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 07:59:15.417886 2266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.134:58921
W20240501 07:59:15.520153 3113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:15.520350 3113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:15.520391 3113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:15.523340 3113 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:15.523414 3113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:15.523516 3113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.135
I20240501 07:59:15.527082 3113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:36633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.0.99.135
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--never_fsync=true
--heap_profile_path=/tmp/kudu.3113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.135
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:15.527866 3113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:15.528959 3113 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:15.536638 3121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:15.537184 3118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:15.537683 3119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:15.538220 3113 server_base.cc:1034] running on GCE node
I20240501 07:59:15.538519 3113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:15.538995 3113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:15.540128 3113 hybrid_clock.cc:648] HybridClock initialized: now 1714550355540106 us; error 35 us; skew 500 ppm
I20240501 07:59:15.541841 3113 webserver.cc:460] Webserver started at http://127.0.99.135:34939/ using document root <none> and password file <none>
I20240501 07:59:15.542326 3113 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:15.542376 3113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:15.542522 3113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:15.544062 3113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "e48105f8f6ab476196dd66c4b12808d8"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:15.544502 3113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "e48105f8f6ab476196dd66c4b12808d8"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:15.547466 3113 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:15.549510 3127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:15.550428 3113 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240501 07:59:15.550518 3113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "e48105f8f6ab476196dd66c4b12808d8"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:15.550593 3113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:15.568341 2455 heartbeater.cc:503] Master 127.0.99.190:35233 was elected leader, sending a full tablet report...
I20240501 07:59:15.595127 3113 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:15.595758 3113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:15.595916 3113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:15.596467 3113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:15.597318 3113 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:15.597362 3113 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:15.597406 3113 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:15.597433 3113 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:15.606211 3113 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.135:37389
I20240501 07:59:15.606245 3240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.135:37389 every 8 connection(s)
I20240501 07:59:15.607056 3113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 07:59:15.613672 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 3113
I20240501 07:59:15.613786 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240501 07:59:15.617141 3241 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35233
I20240501 07:59:15.617362 3241 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:15.617913 3241 heartbeater.cc:511] Master 127.0.99.190:35233 requested a full tablet report, sending...
I20240501 07:59:15.618901 2267 ts_manager.cc:194] Registered new tserver with Master: e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389)
I20240501 07:59:15.619604 2267 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.135:57351
I20240501 07:59:15.630503 398 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240501 07:59:15.640230 2267 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35940:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240501 07:59:15.660035 3044 tablet_service.cc:1467] Processing CreateTablet for tablet cb4496116793436f849ea95bcc6fc6ba (DEFAULT_TABLE table=TestTable [id=8e1fca9f5d8e40cd8da5d523ca3d4e09]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:15.661082 3044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4496116793436f849ea95bcc6fc6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:15.661412 2389 tablet_service.cc:1467] Processing CreateTablet for tablet cb4496116793436f849ea95bcc6fc6ba (DEFAULT_TABLE table=TestTable [id=8e1fca9f5d8e40cd8da5d523ca3d4e09]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:15.662319 2389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4496116793436f849ea95bcc6fc6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:15.665377 2913 tablet_service.cc:1467] Processing CreateTablet for tablet cb4496116793436f849ea95bcc6fc6ba (DEFAULT_TABLE table=TestTable [id=8e1fca9f5d8e40cd8da5d523ca3d4e09]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:15.666208 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4496116793436f849ea95bcc6fc6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:15.665989 2520 tablet_service.cc:1467] Processing CreateTablet for tablet cb4496116793436f849ea95bcc6fc6ba (DEFAULT_TABLE table=TestTable [id=8e1fca9f5d8e40cd8da5d523ca3d4e09]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:15.666765 2520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4496116793436f849ea95bcc6fc6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:15.669080 2651 tablet_service.cc:1467] Processing CreateTablet for tablet cb4496116793436f849ea95bcc6fc6ba (DEFAULT_TABLE table=TestTable [id=8e1fca9f5d8e40cd8da5d523ca3d4e09]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:15.669850 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4496116793436f849ea95bcc6fc6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:15.671312 3263 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:15.673447 3263 tablet_bootstrap.cc:654] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:15.673419 3265 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:15.674387 3263 log.cc:824] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:15.675321 3268 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:15.675966 3264 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:15.676116 3265 tablet_bootstrap.cc:654] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:15.676865 3268 tablet_bootstrap.cc:654] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:15.676846 3265 log.cc:824] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:15.677505 3268 log.cc:824] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:15.677659 3264 tablet_bootstrap.cc:654] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:15.678319 3264 log.cc:824] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:15.678514 3265 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:15.678690 3265 ts_tablet_manager.cc:1367] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20240501 07:59:15.678946 3268 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:15.679111 3268 ts_tablet_manager.cc:1367] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 07:59:15.679798 3264 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:15.679971 3264 ts_tablet_manager.cc:1367] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240501 07:59:15.680333 3266 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:15.681391 3263 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:15.681550 3263 ts_tablet_manager.cc:1367] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20240501 07:59:15.681959 3268 raft_consensus.cc:348] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.682185 3268 raft_consensus.cc:374] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:15.682237 3268 raft_consensus.cc:729] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:15.682701 3268 consensus_queue.cc:260] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.683099 3265 raft_consensus.cc:348] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.683283 3265 raft_consensus.cc:374] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:15.683326 3265 raft_consensus.cc:729] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:15.683280 3264 raft_consensus.cc:348] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.683431 3264 raft_consensus.cc:374] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:15.683468 3264 raft_consensus.cc:729] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:15.683533 3268 ts_tablet_manager.cc:1398] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 07:59:15.683724 3265 consensus_queue.cc:260] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.683820 3264 consensus_queue.cc:260] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.683883 2586 heartbeater.cc:503] Master 127.0.99.190:35233 was elected leader, sending a full tablet report...
I20240501 07:59:15.684556 3265 ts_tablet_manager.cc:1398] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20240501 07:59:15.684556 3264 ts_tablet_manager.cc:1398] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 07:59:15.684876 3110 heartbeater.cc:503] Master 127.0.99.190:35233 was elected leader, sending a full tablet report...
I20240501 07:59:15.684876 2979 heartbeater.cc:503] Master 127.0.99.190:35233 was elected leader, sending a full tablet report...
I20240501 07:59:15.685655 3263 raft_consensus.cc:348] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.685894 3263 raft_consensus.cc:374] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:15.686237 3263 raft_consensus.cc:729] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:15.686676 3263 consensus_queue.cc:260] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.687462 3263 ts_tablet_manager.cc:1398] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 07:59:15.689815 3266 tablet_bootstrap.cc:654] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:15.690464 3266 log.cc:824] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:15.692003 3266 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:15.692158 3266 ts_tablet_manager.cc:1367] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20240501 07:59:15.694384 3266 raft_consensus.cc:348] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.694564 3266 raft_consensus.cc:374] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:15.694620 3266 raft_consensus.cc:729] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:15.695000 3266 consensus_queue.cc:260] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.695583 2717 heartbeater.cc:503] Master 127.0.99.190:35233 was elected leader, sending a full tablet report...
I20240501 07:59:15.695739 3266 ts_tablet_manager.cc:1398] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
W20240501 07:59:15.720669 2980 tablet.cc:2369] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 07:59:15.723405 2587 tablet.cc:2369] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 07:59:15.777287 3274 raft_consensus.cc:482] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:15.777499 3274 raft_consensus.cc:504] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.781375 3274 leader_election.cc:288] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:15.785528 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4496116793436f849ea95bcc6fc6ba" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" is_pre_election: true
I20240501 07:59:15.785878 2671 raft_consensus.cc:2454] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:15.786036 2409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4496116793436f849ea95bcc6fc6ba" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691" is_pre_election: true
I20240501 07:59:15.786036 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4496116793436f849ea95bcc6fc6ba" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0" is_pre_election: true
I20240501 07:59:15.786293 2409 raft_consensus.cc:2454] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:15.786293 3064 raft_consensus.cc:2454] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:15.786666 2869 leader_election.cc:302] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 5dd2fd1701284a6b886ab8d14d1bd8df, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:15.786958 3274 raft_consensus.cc:2781] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:15.787029 3274 raft_consensus.cc:482] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:15.787068 3274 raft_consensus.cc:3037] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:15.787916 3274 raft_consensus.cc:504] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.788415 3274 leader_election.cc:288] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 election: Requested vote from peers 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:15.788697 2409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4496116793436f849ea95bcc6fc6ba" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691"
I20240501 07:59:15.788700 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4496116793436f849ea95bcc6fc6ba" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0"
I20240501 07:59:15.788812 2409 raft_consensus.cc:3037] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:15.788825 3064 raft_consensus.cc:3037] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:15.788831 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4496116793436f849ea95bcc6fc6ba" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
I20240501 07:59:15.788931 2671 raft_consensus.cc:3037] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:15.789888 2671 raft_consensus.cc:2454] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:15.789995 2409 raft_consensus.cc:2454] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:15.790405 2868 leader_election.cc:302] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 5dd2fd1701284a6b886ab8d14d1bd8df, 85377a976c05408fbb384237a0edb691, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:15.790424 3064 raft_consensus.cc:2454] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:15.790498 2539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4496116793436f849ea95bcc6fc6ba" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:15.790565 3274 raft_consensus.cc:2781] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:15.790490 2540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4496116793436f849ea95bcc6fc6ba" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:15.790709 2539 raft_consensus.cc:3037] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:15.790768 3274 raft_consensus.cc:686] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Becoming Leader. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Running, Role: LEADER
I20240501 07:59:15.791107 3274 consensus_queue.cc:237] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:15.791678 2539 raft_consensus.cc:2454] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:15.794328 2267 catalog_manager.cc:5484] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:15.802728 398 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240501 07:59:15.803990 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 85377a976c05408fbb384237a0edb691 to finish bootstrapping
W20240501 07:59:15.806283 2456 tablet.cc:2369] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 07:59:15.814102 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 5810fd56ce9244438edcd3c0ea23e72c to finish bootstrapping
I20240501 07:59:15.818583 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 5dd2fd1701284a6b886ab8d14d1bd8df to finish bootstrapping
I20240501 07:59:15.823139 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 3bd6898f790b48d593253180b79c5e40 to finish bootstrapping
I20240501 07:59:15.828795 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver e2c508eb3dac4193a1eeba799e5d54f2 to finish bootstrapping
I20240501 07:59:15.833382 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 1537338fbeea47ee81f7a210cd6df2f0 to finish bootstrapping
I20240501 07:59:15.838034 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver e48105f8f6ab476196dd66c4b12808d8 to finish bootstrapping
I20240501 07:59:15.843394 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 3113
W20240501 07:59:15.892081 2718 tablet.cc:2369] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 07:59:15.908888 3111 tablet.cc:2369] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 07:59:16.070866 2848 heartbeater.cc:503] Master 127.0.99.190:35233 was elected leader, sending a full tablet report...
I20240501 07:59:16.245959 3274 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:16.260521 3274 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:16.265823 3274 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:16.266460 3293 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.599354 2266 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35940:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:19.606143 2520 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcfbbe5535f45ebad6733432d0f284e (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.606143 2913 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcfbbe5535f45ebad6733432d0f284e (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.606443 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcfbbe5535f45ebad6733432d0f284e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.606443 2520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcfbbe5535f45ebad6733432d0f284e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.606827 2389 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcfbbe5535f45ebad6733432d0f284e (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.607101 2389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcfbbe5535f45ebad6733432d0f284e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.609125 3044 tablet_service.cc:1467] Processing CreateTablet for tablet 902e7d877089496689096005187548fe (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.609395 3044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 902e7d877089496689096005187548fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.609586 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcfbbe5535f45ebad6733432d0f284e (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.609841 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcfbbe5535f45ebad6733432d0f284e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.610275 2520 tablet_service.cc:1467] Processing CreateTablet for tablet 902e7d877089496689096005187548fe (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.610500 2520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 902e7d877089496689096005187548fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.611088 3312 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:19.612018 3312 tablet_bootstrap.cc:654] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.612038 2519 tablet_service.cc:1467] Processing CreateTablet for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.612278 2519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.609598 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 902e7d877089496689096005187548fe (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.609133 3043 tablet_service.cc:1467] Processing CreateTablet for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.612468 3043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.612468 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 902e7d877089496689096005187548fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.612692 3313 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:19.613619 3313 tablet_bootstrap.cc:654] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.614620 3314 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:19.614826 3313 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:19.614902 3313 ts_tablet_manager.cc:1367] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:19.615176 2913 tablet_service.cc:1467] Processing CreateTablet for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.615412 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.615471 3314 tablet_bootstrap.cc:654] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.615427 3313 raft_consensus.cc:348] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.615542 3313 raft_consensus.cc:374] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.615578 3313 raft_consensus.cc:729] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.615695 3313 consensus_queue.cc:260] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.615975 3315 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:19.616035 3313 ts_tablet_manager.cc:1398] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:19.616384 2388 tablet_service.cc:1467] Processing CreateTablet for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.616487 3312 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:19.616559 3312 ts_tablet_manager.cc:1367] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240501 07:59:19.616623 2388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.617017 3312 raft_consensus.cc:348] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.617111 3312 raft_consensus.cc:374] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.617137 3312 raft_consensus.cc:729] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.617249 3312 consensus_queue.cc:260] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.617532 3312 ts_tablet_manager.cc:1398] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:19.617678 3312 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:19.617905 2389 tablet_service.cc:1467] Processing CreateTablet for tablet 902e7d877089496689096005187548fe (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.618098 2389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 902e7d877089496689096005187548fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.618475 3313 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:19.618697 3312 tablet_bootstrap.cc:654] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.618777 2782 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcfbbe5535f45ebad6733432d0f284e (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.619256 3313 tablet_bootstrap.cc:654] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.619709 2782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcfbbe5535f45ebad6733432d0f284e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.620218 3314 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:19.620187 2781 tablet_service.cc:1467] Processing CreateTablet for tablet 902e7d877089496689096005187548fe (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.620296 3314 ts_tablet_manager.cc:1367] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240501 07:59:19.620591 2780 tablet_service.cc:1467] Processing CreateTablet for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa (DEFAULT_TABLE table=rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:19.620784 3314 raft_consensus.cc:348] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.620877 3314 raft_consensus.cc:374] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.620906 3314 raft_consensus.cc:729] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.621016 3314 consensus_queue.cc:260] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.621115 3312 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:19.621184 3312 ts_tablet_manager.cc:1367] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240501 07:59:19.621255 3313 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:19.621320 3313 ts_tablet_manager.cc:1367] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:19.621320 3314 ts_tablet_manager.cc:1398] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:19.621472 3314 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:19.621583 3312 raft_consensus.cc:348] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.621683 3312 raft_consensus.cc:374] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.621713 3312 raft_consensus.cc:729] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.621671 3313 raft_consensus.cc:348] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.621755 3313 raft_consensus.cc:374] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.621781 3313 raft_consensus.cc:729] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.621822 3312 consensus_queue.cc:260] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.621887 3313 consensus_queue.cc:260] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.622143 3312 ts_tablet_manager.cc:1398] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:19.622143 3313 ts_tablet_manager.cc:1398] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:19.622274 3312 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:19.622274 3313 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:19.622295 3314 tablet_bootstrap.cc:654] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.623039 3313 tablet_bootstrap.cc:654] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.623039 3312 tablet_bootstrap.cc:654] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.616070 3311 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:19.624258 2780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.624930 3311 tablet_bootstrap.cc:654] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.625001 3315 tablet_bootstrap.cc:654] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.625370 3313 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:19.625435 3313 ts_tablet_manager.cc:1367] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:19.625798 3313 raft_consensus.cc:348] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.625888 3313 raft_consensus.cc:374] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.625917 3313 raft_consensus.cc:729] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.626052 3312 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:19.626047 3313 consensus_queue.cc:260] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.626119 3312 ts_tablet_manager.cc:1367] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 07:59:19.626353 3313 ts_tablet_manager.cc:1398] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:19.626494 3312 raft_consensus.cc:348] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.626610 3312 raft_consensus.cc:374] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.626637 3312 raft_consensus.cc:729] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.626735 3312 consensus_queue.cc:260] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.626979 3312 ts_tablet_manager.cc:1398] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:19.627178 2781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 902e7d877089496689096005187548fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:19.630259 3330 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:19.630663 3314 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:19.630748 3314 ts_tablet_manager.cc:1367] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240501 07:59:19.631173 3314 raft_consensus.cc:348] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.631281 3314 raft_consensus.cc:374] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.631321 3314 raft_consensus.cc:729] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.631440 3314 consensus_queue.cc:260] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.631690 3314 ts_tablet_manager.cc:1398] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:19.632321 3330 tablet_bootstrap.cc:654] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.632834 3315 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:19.632920 3315 ts_tablet_manager.cc:1367] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240501 07:59:19.633303 3311 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:19.633390 3311 ts_tablet_manager.cc:1367] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240501 07:59:19.633419 3330 log.cc:824] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:19.633423 3315 raft_consensus.cc:348] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.633522 3315 raft_consensus.cc:374] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.633549 3315 raft_consensus.cc:729] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.633661 3315 consensus_queue.cc:260] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.633955 3315 ts_tablet_manager.cc:1398] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:19.633944 3311 raft_consensus.cc:348] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.634040 3311 raft_consensus.cc:374] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.634070 3311 raft_consensus.cc:729] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.634114 3315 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:19.634203 3311 consensus_queue.cc:260] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.634639 3311 ts_tablet_manager.cc:1398] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:19.634778 3311 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:19.634948 3315 tablet_bootstrap.cc:654] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.634966 3330 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:19.635111 3330 ts_tablet_manager.cc:1367] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240501 07:59:19.635550 3311 tablet_bootstrap.cc:654] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.636332 3315 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:19.636497 3315 ts_tablet_manager.cc:1367] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:19.636931 3315 raft_consensus.cc:348] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.637087 3315 raft_consensus.cc:374] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.637157 3315 raft_consensus.cc:729] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.637351 3315 consensus_queue.cc:260] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.637647 3315 ts_tablet_manager.cc:1398] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:19.637718 3311 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:19.637802 3311 ts_tablet_manager.cc:1367] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:19.638046 3330 raft_consensus.cc:348] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.638202 3330 raft_consensus.cc:374] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.638201 3311 raft_consensus.cc:348] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.638250 3330 raft_consensus.cc:729] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.638311 3311 raft_consensus.cc:374] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.638348 3311 raft_consensus.cc:729] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.638460 3311 consensus_queue.cc:260] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.638698 3311 ts_tablet_manager.cc:1398] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:19.638684 3330 consensus_queue.cc:260] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.639521 3330 ts_tablet_manager.cc:1398] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240501 07:59:19.639743 3330 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:19.640529 3330 tablet_bootstrap.cc:654] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.641458 3330 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:19.641523 3330 ts_tablet_manager.cc:1367] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:19.641896 3330 raft_consensus.cc:348] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.641976 3330 raft_consensus.cc:374] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.642006 3330 raft_consensus.cc:729] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.642114 3330 consensus_queue.cc:260] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.642314 3330 ts_tablet_manager.cc:1398] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:19.642444 3330 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:19.643056 3330 tablet_bootstrap.cc:654] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:19.643959 3330 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:19.644038 3330 ts_tablet_manager.cc:1367] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:19.644444 3330 raft_consensus.cc:348] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.644539 3330 raft_consensus.cc:374] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:19.644570 3330 raft_consensus.cc:729] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:19.644672 3330 consensus_queue.cc:260] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.644932 3330 ts_tablet_manager.cc:1398] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:19.661644 3324 raft_consensus.cc:482] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:19.661744 3324 raft_consensus.cc:504] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.662779 3324 leader_election.cc:288] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591)
I20240501 07:59:19.665598 3323 raft_consensus.cc:482] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:19.665701 3323 raft_consensus.cc:504] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.667040 3323 leader_election.cc:288] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519)
I20240501 07:59:19.668092 2802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcfbbe5535f45ebad6733432d0f284e" candidate_uuid: "5810fd56ce9244438edcd3c0ea23e72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:19.668359 2802 raft_consensus.cc:2454] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5810fd56ce9244438edcd3c0ea23e72c in term 0.
I20240501 07:59:19.668552 2933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcfbbe5535f45ebad6733432d0f284e" candidate_uuid: "5810fd56ce9244438edcd3c0ea23e72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" is_pre_election: true
I20240501 07:59:19.668748 2933 raft_consensus.cc:2454] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5810fd56ce9244438edcd3c0ea23e72c in term 0.
I20240501 07:59:19.669159 2475 leader_election.cc:302] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 3bd6898f790b48d593253180b79c5e40, 5810fd56ce9244438edcd3c0ea23e72c, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:19.669178 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcfbbe5535f45ebad6733432d0f284e" candidate_uuid: "5810fd56ce9244438edcd3c0ea23e72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" is_pre_election: true
I20240501 07:59:19.669294 2671 raft_consensus.cc:2454] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5810fd56ce9244438edcd3c0ea23e72c in term 0.
I20240501 07:59:19.669466 3324 raft_consensus.cc:2781] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:19.669530 3324 raft_consensus.cc:482] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:19.669507 3318 raft_consensus.cc:482] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:19.669576 3324 raft_consensus.cc:3037] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.669608 3318 raft_consensus.cc:504] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.670589 3324 raft_consensus.cc:504] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.670789 3318 leader_election.cc:288] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:19.671037 3324 leader_election.cc:288] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [CANDIDATE]: Term 1 election: Requested vote from peers 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591)
I20240501 07:59:19.671679 2802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcfbbe5535f45ebad6733432d0f284e" candidate_uuid: "5810fd56ce9244438edcd3c0ea23e72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:19.671792 2802 raft_consensus.cc:3037] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.672255 2933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcfbbe5535f45ebad6733432d0f284e" candidate_uuid: "5810fd56ce9244438edcd3c0ea23e72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2"
I20240501 07:59:19.672363 2933 raft_consensus.cc:3037] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.672662 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcfbbe5535f45ebad6733432d0f284e" candidate_uuid: "5810fd56ce9244438edcd3c0ea23e72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
I20240501 07:59:19.672761 2671 raft_consensus.cc:3037] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.672825 2802 raft_consensus.cc:2454] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5810fd56ce9244438edcd3c0ea23e72c in term 1.
I20240501 07:59:19.673262 2933 raft_consensus.cc:2454] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5810fd56ce9244438edcd3c0ea23e72c in term 1.
I20240501 07:59:19.673652 2671 raft_consensus.cc:2454] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5810fd56ce9244438edcd3c0ea23e72c in term 1.
I20240501 07:59:19.673641 2475 leader_election.cc:302] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 3bd6898f790b48d593253180b79c5e40, 5810fd56ce9244438edcd3c0ea23e72c, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:19.673941 3324 raft_consensus.cc:2781] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:19.673966 2539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "902e7d877089496689096005187548fe" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:19.674024 3324 raft_consensus.cc:686] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 LEADER]: Becoming Leader. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Running, Role: LEADER
I20240501 07:59:19.674091 2539 raft_consensus.cc:2454] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 0.
I20240501 07:59:19.674330 3324 consensus_queue.cc:237] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:19.677035 2267 catalog_manager.cc:5484] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130). New cstate: current_term: 1 leader_uuid: "5810fd56ce9244438edcd3c0ea23e72c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:19.677728 2409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "902e7d877089496689096005187548fe" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691" is_pre_election: true
I20240501 07:59:19.677868 2409 raft_consensus.cc:2454] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 0.
I20240501 07:59:19.678354 2802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "902e7d877089496689096005187548fe" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:19.678328 2999 leader_election.cc:302] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 5810fd56ce9244438edcd3c0ea23e72c, 85377a976c05408fbb384237a0edb691; no voters:
I20240501 07:59:19.678452 2802 raft_consensus.cc:2454] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 0.
I20240501 07:59:19.678704 3323 raft_consensus.cc:2781] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:19.678763 3323 raft_consensus.cc:482] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:19.678802 3323 raft_consensus.cc:3037] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.679483 2409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcfbbe5535f45ebad6733432d0f284e" candidate_uuid: "5810fd56ce9244438edcd3c0ea23e72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691" is_pre_election: true
I20240501 07:59:19.679600 2409 raft_consensus.cc:2454] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5810fd56ce9244438edcd3c0ea23e72c in term 0.
I20240501 07:59:19.679716 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fcfbbe5535f45ebad6733432d0f284e" candidate_uuid: "5810fd56ce9244438edcd3c0ea23e72c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691"
I20240501 07:59:19.679713 3323 raft_consensus.cc:504] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.679821 2408 raft_consensus.cc:3037] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.680166 3323 leader_election.cc:288] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [CANDIDATE]: Term 1 election: Requested vote from peers 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519)
I20240501 07:59:19.680502 2802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "902e7d877089496689096005187548fe" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:19.680588 2802 raft_consensus.cc:3037] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.680737 2408 raft_consensus.cc:2454] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5810fd56ce9244438edcd3c0ea23e72c in term 1.
I20240501 07:59:19.680907 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "902e7d877089496689096005187548fe" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" is_pre_election: true
I20240501 07:59:19.681022 2671 raft_consensus.cc:2454] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 0.
I20240501 07:59:19.681385 2802 raft_consensus.cc:2454] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 1.
I20240501 07:59:19.681824 2409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "902e7d877089496689096005187548fe" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691"
I20240501 07:59:19.681921 2409 raft_consensus.cc:3037] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.682068 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "902e7d877089496689096005187548fe" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
I20240501 07:59:19.682164 2671 raft_consensus.cc:3037] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.682708 2409 raft_consensus.cc:2454] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 1.
I20240501 07:59:19.682968 2671 raft_consensus.cc:2454] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 1.
I20240501 07:59:19.682986 2999 leader_election.cc:302] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 3bd6898f790b48d593253180b79c5e40, 85377a976c05408fbb384237a0edb691; no voters:
I20240501 07:59:19.683393 3323 raft_consensus.cc:2781] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:19.683413 2539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "902e7d877089496689096005187548fe" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:19.683516 2539 raft_consensus.cc:3037] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.683588 3323 raft_consensus.cc:686] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 LEADER]: Becoming Leader. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Running, Role: LEADER
I20240501 07:59:19.683854 2933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2bcc9afe2f4c839e80fadb7fcf4dfa" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" is_pre_election: true
I20240501 07:59:19.683872 3323 consensus_queue.cc:237] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:19.683980 2933 raft_consensus.cc:2454] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 0.
I20240501 07:59:19.684470 2539 raft_consensus.cc:2454] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 1.
I20240501 07:59:19.686147 2267 catalog_manager.cc:5484] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134). New cstate: current_term: 1 leader_uuid: "1537338fbeea47ee81f7a210cd6df2f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:19.686635 2802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2bcc9afe2f4c839e80fadb7fcf4dfa" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:19.686736 2802 raft_consensus.cc:2454] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 0.
I20240501 07:59:19.686987 2344 leader_election.cc:302] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 3bd6898f790b48d593253180b79c5e40, 85377a976c05408fbb384237a0edb691, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:19.687220 3318 raft_consensus.cc:2781] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:19.687275 3318 raft_consensus.cc:482] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:19.687319 3318 raft_consensus.cc:3037] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.687327 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2bcc9afe2f4c839e80fadb7fcf4dfa" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0" is_pre_election: true
I20240501 07:59:19.687433 3064 raft_consensus.cc:2454] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 0.
I20240501 07:59:19.688169 3318 raft_consensus.cc:504] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.688306 2539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2bcc9afe2f4c839e80fadb7fcf4dfa" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:19.688400 2539 raft_consensus.cc:2454] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 0.
I20240501 07:59:19.688685 3318 leader_election.cc:288] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [CANDIDATE]: Term 1 election: Requested vote from peers 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:19.688943 2933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2bcc9afe2f4c839e80fadb7fcf4dfa" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2"
I20240501 07:59:19.689011 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2bcc9afe2f4c839e80fadb7fcf4dfa" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0"
I20240501 07:59:19.689047 2933 raft_consensus.cc:3037] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.689103 3064 raft_consensus.cc:3037] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.689132 2802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2bcc9afe2f4c839e80fadb7fcf4dfa" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:19.689213 2802 raft_consensus.cc:3037] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.689656 2933 raft_consensus.cc:2454] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 1.
I20240501 07:59:19.689926 3064 raft_consensus.cc:2454] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 1.
I20240501 07:59:19.689934 2802 raft_consensus.cc:2454] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 1.
I20240501 07:59:19.690059 2539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2bcc9afe2f4c839e80fadb7fcf4dfa" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:19.690155 2539 raft_consensus.cc:3037] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:19.690192 2345 leader_election.cc:302] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 85377a976c05408fbb384237a0edb691, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:19.690361 3318 raft_consensus.cc:2781] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:19.690544 3318 raft_consensus.cc:686] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Becoming Leader. State: Replica: 85377a976c05408fbb384237a0edb691, State: Running, Role: LEADER
I20240501 07:59:19.690824 3318 consensus_queue.cc:237] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:19.690884 2539 raft_consensus.cc:2454] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 1.
I20240501 07:59:19.693042 2267 catalog_manager.cc:5484] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85377a976c05408fbb384237a0edb691 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } }
W20240501 07:59:19.804935 3364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:19.805152 3364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:19.808089 3364 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20240501 07:59:19.811563 2849 tablet.cc:2369] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40: Can't schedule compaction. Clean time has not been advanced past its initial value.
Using table 'rebalance_test_table_0'
I20240501 07:59:19.882805 2539 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:19.882848 2933 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:19.883392 3318 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.884042 3362 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.886888 2671 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:19.887307 3064 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:19.888576 2802 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:19.889022 3323 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.889842 3362 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.890127 3362 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.894196 2409 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:19.894542 2932 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:19.894871 2801 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:19.895048 2540 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:19.895208 2670 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:19.895629 3324 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.896195 3320 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.896481 3320 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.896736 3320 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.897970 3323 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.917450 2802 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:19.917476 2408 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:19.918352 3360 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.918495 3416 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:19.918593 3392 mvcc.cc:205] Tried to move back new op lower bound from 7022798274090123264 to 7022798273268547584. Current Snapshot: MvccSnapshot[applied={T|T < 7022798274090123264}]
I20240501 07:59:19.925201 3409 mvcc.cc:205] Tried to move back new op lower bound from 7022798274118012928 to 7022798273229332480. Current Snapshot: MvccSnapshot[applied={T|T < 7022798274118012928}]
I20240501 07:59:19.927139 3389 mvcc.cc:205] Tried to move back new op lower bound from 7022798274195435520 to 7022798273296527360. Current Snapshot: MvccSnapshot[applied={T|T < 7022798274074251264}]
I20240501 07:59:19.929263 3413 mvcc.cc:205] Tried to move back new op lower bound from 7022798274217492480 to 7022798273229332480. Current Snapshot: MvccSnapshot[applied={T|T < 7022798274118012928}]
I20240501 07:59:19.945179 3399 mvcc.cc:205] Tried to move back new op lower bound from 7022798274074251264 to 7022798273296527360. Current Snapshot: MvccSnapshot[applied={T|T < 7022798274074251264}]
I20240501 07:59:19.983810 3400 mvcc.cc:205] Tried to move back new op lower bound from 7022798274118012928 to 7022798273229332480. Current Snapshot: MvccSnapshot[applied={T|T < 7022798274118012928}]
INSERT report
rows total: 2000
time total: 151.534 ms
time per row: 0.0757668 ms
I20240501 07:59:20.001327 2262 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35940:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:20.006582 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae5c568fb854f2a9e4122fc21314024 (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.006666 2913 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae5c568fb854f2a9e4122fc21314024 (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.006728 2782 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae5c568fb854f2a9e4122fc21314024 (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.006877 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae5c568fb854f2a9e4122fc21314024. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.006949 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae5c568fb854f2a9e4122fc21314024. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.006965 2782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae5c568fb854f2a9e4122fc21314024. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.008285 3043 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae5c568fb854f2a9e4122fc21314024 (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.008539 3043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae5c568fb854f2a9e4122fc21314024. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.008958 3330 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:20.008958 3315 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:20.009124 3042 tablet_service.cc:1467] Processing CreateTablet for tablet 490523aeb9e44f4ba8c2fc42fb73a95e (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.009363 3042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 490523aeb9e44f4ba8c2fc42fb73a95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.009740 3315 tablet_bootstrap.cc:654] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.009740 3330 tablet_bootstrap.cc:654] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.010432 3311 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:20.010608 3044 tablet_service.cc:1467] Processing CreateTablet for tablet 917c88788fbf4089a5df0f271cc478f7 (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.010808 3044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917c88788fbf4089a5df0f271cc478f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.011160 3311 tablet_bootstrap.cc:654] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.012058 3314 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:20.012167 3311 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:20.012240 3311 ts_tablet_manager.cc:1367] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:20.012207 2520 tablet_service.cc:1467] Processing CreateTablet for tablet 917c88788fbf4089a5df0f271cc478f7 (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.012455 2520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917c88788fbf4089a5df0f271cc478f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.012679 3311 raft_consensus.cc:348] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.012787 3311 raft_consensus.cc:374] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.012835 3311 raft_consensus.cc:729] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.012897 3314 tablet_bootstrap.cc:654] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.012948 3311 consensus_queue.cc:260] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.013203 3311 ts_tablet_manager.cc:1398] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.013621 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 490523aeb9e44f4ba8c2fc42fb73a95e (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.013866 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 490523aeb9e44f4ba8c2fc42fb73a95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.014266 2389 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae5c568fb854f2a9e4122fc21314024 (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.014534 2389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae5c568fb854f2a9e4122fc21314024. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.014503 2782 tablet_service.cc:1467] Processing CreateTablet for tablet 917c88788fbf4089a5df0f271cc478f7 (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.014640 3312 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:20.014722 2782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917c88788fbf4089a5df0f271cc478f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.015421 3312 tablet_bootstrap.cc:654] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.014535 2388 tablet_service.cc:1467] Processing CreateTablet for tablet 490523aeb9e44f4ba8c2fc42fb73a95e (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.016140 2388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 490523aeb9e44f4ba8c2fc42fb73a95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.016176 2780 tablet_service.cc:1467] Processing CreateTablet for tablet 490523aeb9e44f4ba8c2fc42fb73a95e (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.016388 2780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 490523aeb9e44f4ba8c2fc42fb73a95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.016479 3313 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:20.017284 3313 tablet_bootstrap.cc:654] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.018312 3315 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:20.018378 3313 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:20.018414 3315 ts_tablet_manager.cc:1367] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240501 07:59:20.018441 3313 ts_tablet_manager.cc:1367] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:20.018827 3313 raft_consensus.cc:348] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.018827 3315 raft_consensus.cc:348] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.018941 3313 raft_consensus.cc:374] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.018945 3315 raft_consensus.cc:374] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.018975 3315 raft_consensus.cc:729] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.018975 3313 raft_consensus.cc:729] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.019083 3313 consensus_queue.cc:260] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.019083 3315 consensus_queue.cc:260] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.019244 2913 tablet_service.cc:1467] Processing CreateTablet for tablet 917c88788fbf4089a5df0f271cc478f7 (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.019331 3313 ts_tablet_manager.cc:1398] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.019331 3315 ts_tablet_manager.cc:1398] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.019483 3313 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:20.019483 3315 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:20.019484 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917c88788fbf4089a5df0f271cc478f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.020251 2912 tablet_service.cc:1467] Processing CreateTablet for tablet 490523aeb9e44f4ba8c2fc42fb73a95e (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.020311 3313 tablet_bootstrap.cc:654] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.020489 2912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 490523aeb9e44f4ba8c2fc42fb73a95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.021873 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 917c88788fbf4089a5df0f271cc478f7 (DEFAULT_TABLE table=rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.022091 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917c88788fbf4089a5df0f271cc478f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.022511 3311 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:20.023291 3315 tablet_bootstrap.cc:654] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.023685 3311 tablet_bootstrap.cc:654] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.024361 3315 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:20.024442 3315 ts_tablet_manager.cc:1367] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 07:59:20.025130 3311 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:20.025214 3311 ts_tablet_manager.cc:1367] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:20.025269 3315 raft_consensus.cc:348] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.025382 3315 raft_consensus.cc:374] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.025416 3315 raft_consensus.cc:729] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.025435 3312 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:20.025518 3312 ts_tablet_manager.cc:1367] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240501 07:59:20.025637 3311 raft_consensus.cc:348] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.025741 3311 raft_consensus.cc:374] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.025775 3311 raft_consensus.cc:729] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.025892 3311 consensus_queue.cc:260] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.025982 3314 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:20.025972 3312 raft_consensus.cc:348] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.026067 3314 ts_tablet_manager.cc:1367] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20240501 07:59:20.026085 3312 raft_consensus.cc:374] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.026119 3312 raft_consensus.cc:729] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.026160 3311 ts_tablet_manager.cc:1398] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:20.026237 3312 consensus_queue.cc:260] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.026315 3311 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:20.026479 3312 ts_tablet_manager.cc:1398] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:20.026484 3314 raft_consensus.cc:348] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.026598 3314 raft_consensus.cc:374] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.026636 3314 raft_consensus.cc:729] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.026760 3314 consensus_queue.cc:260] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.027012 3314 ts_tablet_manager.cc:1398] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:20.027151 3311 tablet_bootstrap.cc:654] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.027204 3314 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:20.028046 3314 tablet_bootstrap.cc:654] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.028388 3313 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:20.028470 3311 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:20.028473 3313 ts_tablet_manager.cc:1367] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240501 07:59:20.028537 3311 ts_tablet_manager.cc:1367] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:20.028918 3330 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:20.028924 3311 raft_consensus.cc:348] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.029002 3330 ts_tablet_manager.cc:1367] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20240501 07:59:20.029011 3311 raft_consensus.cc:374] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.028956 3313 raft_consensus.cc:348] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.029039 3311 raft_consensus.cc:729] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.029062 3313 raft_consensus.cc:374] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.029093 3313 raft_consensus.cc:729] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.029141 3311 consensus_queue.cc:260] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.029203 3313 consensus_queue.cc:260] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.029450 3311 ts_tablet_manager.cc:1398] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.029467 3313 ts_tablet_manager.cc:1398] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.029444 3330 raft_consensus.cc:348] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.029542 3330 raft_consensus.cc:374] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.029573 3330 raft_consensus.cc:729] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.029670 3330 consensus_queue.cc:260] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.029949 3330 ts_tablet_manager.cc:1398] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.030089 3330 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:20.030895 3330 tablet_bootstrap.cc:654] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.031834 3330 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:20.031940 3330 ts_tablet_manager.cc:1367] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:20.025547 3315 consensus_queue.cc:260] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.032414 3315 ts_tablet_manager.cc:1398] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240501 07:59:20.032693 3315 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:20.033252 3330 raft_consensus.cc:348] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.033353 3330 raft_consensus.cc:374] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.033388 3330 raft_consensus.cc:729] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.033505 3330 consensus_queue.cc:260] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.033799 3315 tablet_bootstrap.cc:654] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.034461 3330 ts_tablet_manager.cc:1398] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:20.034622 3330 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:20.034909 3314 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:20.034996 3314 ts_tablet_manager.cc:1367] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240501 07:59:20.035454 3330 tablet_bootstrap.cc:654] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.035411 3314 raft_consensus.cc:348] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.035511 3314 raft_consensus.cc:374] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.035547 3314 raft_consensus.cc:729] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.035674 3314 consensus_queue.cc:260] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.036386 3330 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:20.036446 3314 ts_tablet_manager.cc:1398] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.036455 3330 ts_tablet_manager.cc:1367] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:20.036610 3314 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:20.036821 3330 raft_consensus.cc:348] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.036903 3330 raft_consensus.cc:374] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.036931 3330 raft_consensus.cc:729] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.037035 3330 consensus_queue.cc:260] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.037343 3330 ts_tablet_manager.cc:1398] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.037489 3314 tablet_bootstrap.cc:654] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.038518 3315 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:20.038597 3315 ts_tablet_manager.cc:1367] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 07:59:20.038919 3315 raft_consensus.cc:348] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.039002 3315 raft_consensus.cc:374] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.039028 3315 raft_consensus.cc:729] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.039145 3315 consensus_queue.cc:260] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.039410 3315 ts_tablet_manager.cc:1398] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:20.039784 3314 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:20.039866 3314 ts_tablet_manager.cc:1367] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:20.040264 3314 raft_consensus.cc:348] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.040350 3314 raft_consensus.cc:374] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.040378 3314 raft_consensus.cc:729] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.040504 3314 consensus_queue.cc:260] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.040746 3314 ts_tablet_manager.cc:1398] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.044044 3362 raft_consensus.cc:482] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:20.044131 3362 raft_consensus.cc:504] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.044643 3362 leader_election.cc:288] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027)
I20240501 07:59:20.045083 2801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae5c568fb854f2a9e4122fc21314024" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:20.045192 2801 raft_consensus.cc:2454] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 0.
I20240501 07:59:20.045176 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae5c568fb854f2a9e4122fc21314024" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0" is_pre_election: true
I20240501 07:59:20.045284 3064 raft_consensus.cc:2454] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 0.
I20240501 07:59:20.045506 2933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae5c568fb854f2a9e4122fc21314024" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" is_pre_election: true
I20240501 07:59:20.045635 2933 raft_consensus.cc:2454] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 0.
I20240501 07:59:20.045842 2345 leader_election.cc:302] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 3bd6898f790b48d593253180b79c5e40, 85377a976c05408fbb384237a0edb691; no voters:
I20240501 07:59:20.046012 3362 raft_consensus.cc:2781] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:20.046063 3362 raft_consensus.cc:482] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:20.046087 3362 raft_consensus.cc:3037] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.046816 3362 raft_consensus.cc:504] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.047214 3362 leader_election.cc:288] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [CANDIDATE]: Term 1 election: Requested vote from peers 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027)
I20240501 07:59:20.047524 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae5c568fb854f2a9e4122fc21314024" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0"
I20240501 07:59:20.047621 3064 raft_consensus.cc:3037] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.048074 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae5c568fb854f2a9e4122fc21314024" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" is_pre_election: true
I20240501 07:59:20.048110 2933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae5c568fb854f2a9e4122fc21314024" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2"
I20240501 07:59:20.048122 2801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae5c568fb854f2a9e4122fc21314024" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:20.048202 2933 raft_consensus.cc:3037] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.048202 2801 raft_consensus.cc:3037] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.048192 2670 raft_consensus.cc:2454] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 0.
I20240501 07:59:20.048357 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae5c568fb854f2a9e4122fc21314024" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
I20240501 07:59:20.048430 2671 raft_consensus.cc:3037] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.048496 3064 raft_consensus.cc:2454] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 1.
I20240501 07:59:20.049005 2933 raft_consensus.cc:2454] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 1.
I20240501 07:59:20.049316 2671 raft_consensus.cc:2454] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 1.
I20240501 07:59:20.049343 2801 raft_consensus.cc:2454] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 1.
I20240501 07:59:20.049551 2344 leader_election.cc:302] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 3bd6898f790b48d593253180b79c5e40, 85377a976c05408fbb384237a0edb691; no voters:
I20240501 07:59:20.049775 3362 raft_consensus.cc:2781] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:20.049852 3362 raft_consensus.cc:686] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Becoming Leader. State: Replica: 85377a976c05408fbb384237a0edb691, State: Running, Role: LEADER
I20240501 07:59:20.049978 3362 consensus_queue.cc:237] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:20.051585 2262 catalog_manager.cc:5484] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85377a976c05408fbb384237a0edb691 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:20.185446 3304 raft_consensus.cc:482] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:20.185549 3304 raft_consensus.cc:504] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.186077 3304 leader_election.cc:288] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829)
I20240501 07:59:20.186380 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917c88788fbf4089a5df0f271cc478f7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0" is_pre_election: true
I20240501 07:59:20.186527 3064 raft_consensus.cc:2454] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:20.186518 2539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917c88788fbf4089a5df0f271cc478f7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:20.186637 2539 raft_consensus.cc:2454] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:20.186766 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917c88788fbf4089a5df0f271cc478f7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" is_pre_election: true
I20240501 07:59:20.186913 2671 raft_consensus.cc:2454] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:20.186976 2866 leader_election.cc:302] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 5810fd56ce9244438edcd3c0ea23e72c, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:20.187129 3304 raft_consensus.cc:2781] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:20.187179 3304 raft_consensus.cc:482] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:20.187214 3304 raft_consensus.cc:3037] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.189563 3304 raft_consensus.cc:504] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.189864 2801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917c88788fbf4089a5df0f271cc478f7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:20.189988 2801 raft_consensus.cc:2454] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:20.190042 3304 leader_election.cc:288] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 election: Requested vote from peers 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829)
I20240501 07:59:20.190263 2539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917c88788fbf4089a5df0f271cc478f7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:20.190385 2539 raft_consensus.cc:3037] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.190397 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917c88788fbf4089a5df0f271cc478f7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0"
I20240501 07:59:20.190407 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917c88788fbf4089a5df0f271cc478f7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
I20240501 07:59:20.190495 3064 raft_consensus.cc:3037] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.190495 2671 raft_consensus.cc:3037] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.190691 2801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917c88788fbf4089a5df0f271cc478f7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:20.190775 2801 raft_consensus.cc:3037] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.191293 2539 raft_consensus.cc:2454] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:20.191319 3064 raft_consensus.cc:2454] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:20.191319 2671 raft_consensus.cc:2454] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:20.191601 2801 raft_consensus.cc:2454] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:20.191658 2866 leader_election.cc:302] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 5810fd56ce9244438edcd3c0ea23e72c, 5dd2fd1701284a6b886ab8d14d1bd8df, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:20.191833 3304 raft_consensus.cc:2781] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:20.191990 3304 raft_consensus.cc:686] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Becoming Leader. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Running, Role: LEADER
I20240501 07:59:20.192270 3304 consensus_queue.cc:237] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.193889 2262 catalog_manager.cc:5484] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:20.323455 3332 raft_consensus.cc:482] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:20.323570 3332 raft_consensus.cc:504] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.324698 3332 leader_election.cc:288] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829)
I20240501 07:59:20.328738 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490523aeb9e44f4ba8c2fc42fb73a95e" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691" is_pre_election: true
I20240501 07:59:20.328883 2408 raft_consensus.cc:2454] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:20.328881 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490523aeb9e44f4ba8c2fc42fb73a95e" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0" is_pre_election: true
I20240501 07:59:20.328989 3064 raft_consensus.cc:2454] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:20.329322 2606 leader_election.cc:302] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 5dd2fd1701284a6b886ab8d14d1bd8df, 85377a976c05408fbb384237a0edb691; no voters:
I20240501 07:59:20.329530 3332 raft_consensus.cc:2781] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:20.329586 3332 raft_consensus.cc:482] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:20.329623 3332 raft_consensus.cc:3037] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.330267 2801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490523aeb9e44f4ba8c2fc42fb73a95e" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:20.330420 2801 raft_consensus.cc:2454] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:20.330572 3332 raft_consensus.cc:504] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.331036 3332 leader_election.cc:288] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 election: Requested vote from peers 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829)
I20240501 07:59:20.331313 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490523aeb9e44f4ba8c2fc42fb73a95e" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0"
I20240501 07:59:20.331336 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490523aeb9e44f4ba8c2fc42fb73a95e" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691"
I20240501 07:59:20.331442 3064 raft_consensus.cc:3037] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.331449 2408 raft_consensus.cc:3037] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.331430 2933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490523aeb9e44f4ba8c2fc42fb73a95e" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" is_pre_election: true
I20240501 07:59:20.331537 2933 raft_consensus.cc:2454] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:20.331566 2932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490523aeb9e44f4ba8c2fc42fb73a95e" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2"
I20240501 07:59:20.331662 2932 raft_consensus.cc:3037] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.332233 2408 raft_consensus.cc:2454] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:20.332402 3064 raft_consensus.cc:2454] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:20.332510 2932 raft_consensus.cc:2454] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:20.332700 2607 leader_election.cc:302] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 5dd2fd1701284a6b886ab8d14d1bd8df, 85377a976c05408fbb384237a0edb691; no voters:
I20240501 07:59:20.332830 3332 raft_consensus.cc:2781] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:20.333005 3332 raft_consensus.cc:686] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Becoming Leader. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Running, Role: LEADER
I20240501 07:59:20.333081 2801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "490523aeb9e44f4ba8c2fc42fb73a95e" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:20.333191 2801 raft_consensus.cc:3037] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.333312 3332 consensus_queue.cc:237] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:20.334024 2801 raft_consensus.cc:2454] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:20.335718 2262 catalog_manager.cc:5484] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: term changed from 0 to 1, leader changed from <none> to 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131). New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:20.472299 3362 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 07:59:20.491348 3468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:20.491554 3468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:20.494513 3468 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240501 07:59:20.568279 3064 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.568636 2801 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.568693 3459 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.569023 2671 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.569355 2539 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.570189 3304 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.570463 3304 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.570719 3304 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.579102 3064 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.579234 2802 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.579620 3362 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.579898 3362 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.583814 2932 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.584173 3362 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.588310 2800 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.589102 3332 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.590217 3064 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.590567 3332 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.592218 2408 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.592450 2933 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.592553 3332 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.593467 3332 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 141.486 ms
time per row: 0.0707428 ms
I20240501 07:59:20.677294 2263 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35940:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:20.682193 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 97a598d6cbb84578ab82e3807b98f36f (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.682377 2782 tablet_service.cc:1467] Processing CreateTablet for tablet 97a598d6cbb84578ab82e3807b98f36f (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.682493 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97a598d6cbb84578ab82e3807b98f36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.682601 2782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97a598d6cbb84578ab82e3807b98f36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.682605 2520 tablet_service.cc:1467] Processing CreateTablet for tablet 97a598d6cbb84578ab82e3807b98f36f (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.682897 2520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97a598d6cbb84578ab82e3807b98f36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.684000 2518 tablet_service.cc:1467] Processing CreateTablet for tablet 3ade42f3f20e4deda01e89ee6ee4141f (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.684201 2518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ade42f3f20e4deda01e89ee6ee4141f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.684782 2519 tablet_service.cc:1467] Processing CreateTablet for tablet 382e9b7922f849f980b80ded29e99075 (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.684804 2388 tablet_service.cc:1467] Processing CreateTablet for tablet 97a598d6cbb84578ab82e3807b98f36f (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.685014 2388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97a598d6cbb84578ab82e3807b98f36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.685012 2519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 382e9b7922f849f980b80ded29e99075. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.686228 3042 tablet_service.cc:1467] Processing CreateTablet for tablet 3ade42f3f20e4deda01e89ee6ee4141f (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.686508 3042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ade42f3f20e4deda01e89ee6ee4141f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.686833 3556 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:20.687060 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 382e9b7922f849f980b80ded29e99075 (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.687291 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 382e9b7922f849f980b80ded29e99075. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.688165 3556 tablet_bootstrap.cc:654] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.688189 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 3ade42f3f20e4deda01e89ee6ee4141f (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.688406 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ade42f3f20e4deda01e89ee6ee4141f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.688400 2782 tablet_service.cc:1467] Processing CreateTablet for tablet 382e9b7922f849f980b80ded29e99075 (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.688618 2782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 382e9b7922f849f980b80ded29e99075. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.686228 3044 tablet_service.cc:1467] Processing CreateTablet for tablet 382e9b7922f849f980b80ded29e99075 (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.689313 3044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 382e9b7922f849f980b80ded29e99075. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.689460 2912 tablet_service.cc:1467] Processing CreateTablet for tablet 3ade42f3f20e4deda01e89ee6ee4141f (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.689699 2912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ade42f3f20e4deda01e89ee6ee4141f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.690268 2913 tablet_service.cc:1467] Processing CreateTablet for tablet 97a598d6cbb84578ab82e3807b98f36f (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.690474 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97a598d6cbb84578ab82e3807b98f36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.691802 2389 tablet_service.cc:1467] Processing CreateTablet for tablet 3ade42f3f20e4deda01e89ee6ee4141f (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.692059 2389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ade42f3f20e4deda01e89ee6ee4141f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.692039 2388 tablet_service.cc:1467] Processing CreateTablet for tablet 382e9b7922f849f980b80ded29e99075 (DEFAULT_TABLE table=rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:20.692217 2388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 382e9b7922f849f980b80ded29e99075. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:20.693025 3561 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:20.693212 3556 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:20.693296 3556 ts_tablet_manager.cc:1367] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240501 07:59:20.693740 3556 raft_consensus.cc:348] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.693861 3556 raft_consensus.cc:374] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.693903 3556 raft_consensus.cc:729] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.694007 3561 tablet_bootstrap.cc:654] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.694032 3556 consensus_queue.cc:260] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.694142 3560 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:20.694312 3556 ts_tablet_manager.cc:1398] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.694741 3558 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:20.694769 3556 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:20.695040 3560 tablet_bootstrap.cc:654] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.695549 3556 tablet_bootstrap.cc:654] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.695559 3558 tablet_bootstrap.cc:654] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.695587 3555 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:20.696502 3555 tablet_bootstrap.cc:654] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.696521 3560 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:20.696537 3556 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:20.696593 3556 ts_tablet_manager.cc:1367] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:20.696593 3560 ts_tablet_manager.cc:1367] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 07:59:20.696995 3556 raft_consensus.cc:348] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.696995 3560 raft_consensus.cc:348] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.697110 3560 raft_consensus.cc:374] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.697110 3556 raft_consensus.cc:374] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.697142 3556 raft_consensus.cc:729] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.697142 3560 raft_consensus.cc:729] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.697270 3560 consensus_queue.cc:260] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.697482 3555 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:20.697540 3560 ts_tablet_manager.cc:1398] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:20.697561 3555 ts_tablet_manager.cc:1367] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 07:59:20.697705 3560 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:20.697924 3555 raft_consensus.cc:348] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.698019 3555 raft_consensus.cc:374] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.698047 3555 raft_consensus.cc:729] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.698172 3555 consensus_queue.cc:260] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.698498 3555 ts_tablet_manager.cc:1398] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:20.698525 3560 tablet_bootstrap.cc:654] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.698648 3555 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:20.699216 3557 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:20.699712 3555 tablet_bootstrap.cc:654] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.699779 3556 consensus_queue.cc:260] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.700134 3556 ts_tablet_manager.cc:1398] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:20.700290 3556 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:20.700318 3561 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:20.700385 3561 ts_tablet_manager.cc:1367] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240501 07:59:20.700778 3561 raft_consensus.cc:348] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.700876 3561 raft_consensus.cc:374] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.700917 3561 raft_consensus.cc:729] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.700979 3560 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:20.701081 3556 tablet_bootstrap.cc:654] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.701100 3560 ts_tablet_manager.cc:1367] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:20.701056 3561 consensus_queue.cc:260] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.701323 3561 ts_tablet_manager.cc:1398] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.701452 3561 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:20.701666 3560 raft_consensus.cc:348] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.701762 3560 raft_consensus.cc:374] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.701792 3560 raft_consensus.cc:729] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.701915 3560 consensus_queue.cc:260] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.702252 3560 ts_tablet_manager.cc:1398] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.702286 3561 tablet_bootstrap.cc:654] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.702852 3555 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:20.702950 3555 ts_tablet_manager.cc:1367] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 07:59:20.703192 3557 tablet_bootstrap.cc:654] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.703691 3555 raft_consensus.cc:348] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.703794 3555 raft_consensus.cc:374] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.703826 3555 raft_consensus.cc:729] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.703974 3555 consensus_queue.cc:260] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.704232 3558 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:20.704305 3555 ts_tablet_manager.cc:1398] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.704317 3558 ts_tablet_manager.cc:1367] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240501 07:59:20.704697 3558 raft_consensus.cc:348] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.704790 3558 raft_consensus.cc:374] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.704818 3558 raft_consensus.cc:729] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.704938 3558 consensus_queue.cc:260] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.705209 3558 ts_tablet_manager.cc:1398] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:20.705384 3558 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:20.706316 3558 tablet_bootstrap.cc:654] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.706785 3561 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:20.706861 3561 ts_tablet_manager.cc:1367] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 07:59:20.707021 3557 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:20.707089 3557 ts_tablet_manager.cc:1367] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240501 07:59:20.707253 3561 raft_consensus.cc:348] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.707348 3561 raft_consensus.cc:374] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.707378 3561 raft_consensus.cc:729] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.707521 3558 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:20.707489 3561 consensus_queue.cc:260] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.707502 3557 raft_consensus.cc:348] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.707599 3557 raft_consensus.cc:374] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.707603 3558 ts_tablet_manager.cc:1367] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 07:59:20.707633 3557 raft_consensus.cc:729] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.707729 3561 ts_tablet_manager.cc:1398] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.708003 3556 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:20.708011 3558 raft_consensus.cc:348] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.708108 3558 raft_consensus.cc:374] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.708139 3558 raft_consensus.cc:729] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.708262 3556 ts_tablet_manager.cc:1367] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240501 07:59:20.708250 3558 consensus_queue.cc:260] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.708489 3558 ts_tablet_manager.cc:1398] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.708628 3558 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:20.709422 3558 tablet_bootstrap.cc:654] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.709874 3556 raft_consensus.cc:348] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.710033 3556 raft_consensus.cc:374] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.710107 3556 raft_consensus.cc:729] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.707777 3557 consensus_queue.cc:260] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.710443 3558 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:20.710496 3557 ts_tablet_manager.cc:1398] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:20.710510 3558 ts_tablet_manager.cc:1367] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:20.710614 3557 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:20.710875 3558 raft_consensus.cc:348] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.710979 3558 raft_consensus.cc:374] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.711011 3558 raft_consensus.cc:729] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.711113 3558 consensus_queue.cc:260] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.711228 3557 tablet_bootstrap.cc:654] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.711346 3558 ts_tablet_manager.cc:1398] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.711726 3556 consensus_queue.cc:260] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.712085 3556 ts_tablet_manager.cc:1398] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240501 07:59:20.712175 3557 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:20.712244 3557 ts_tablet_manager.cc:1367] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:20.712669 3557 raft_consensus.cc:348] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.712767 3557 raft_consensus.cc:374] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.712798 3557 raft_consensus.cc:729] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.712900 3557 consensus_queue.cc:260] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.713126 3557 ts_tablet_manager.cc:1398] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.713264 3557 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:20.713908 3557 tablet_bootstrap.cc:654] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:20.714953 3557 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:20.715027 3557 ts_tablet_manager.cc:1367] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:20.715400 3557 raft_consensus.cc:348] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.715504 3557 raft_consensus.cc:374] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:20.715538 3557 raft_consensus.cc:729] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:20.715657 3557 consensus_queue.cc:260] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.715981 3557 ts_tablet_manager.cc:1398] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:20.748473 3504 raft_consensus.cc:482] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:20.748612 3504 raft_consensus.cc:504] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.749130 3504 leader_election.cc:288] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:20.749435 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a598d6cbb84578ab82e3807b98f36f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" is_pre_election: true
I20240501 07:59:20.749491 2539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a598d6cbb84578ab82e3807b98f36f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:20.749503 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a598d6cbb84578ab82e3807b98f36f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691" is_pre_election: true
I20240501 07:59:20.749516 2800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a598d6cbb84578ab82e3807b98f36f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:20.749577 2671 raft_consensus.cc:2454] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:20.749616 2539 raft_consensus.cc:2454] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:20.749616 2408 raft_consensus.cc:2454] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:20.749630 2800 raft_consensus.cc:2454] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:20.749904 2866 leader_election.cc:302] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 5810fd56ce9244438edcd3c0ea23e72c, 5dd2fd1701284a6b886ab8d14d1bd8df, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:20.750082 3504 raft_consensus.cc:2781] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:20.750134 3504 raft_consensus.cc:482] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:20.750178 3504 raft_consensus.cc:3037] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.751078 3504 raft_consensus.cc:504] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.751482 3504 leader_election.cc:288] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 election: Requested vote from peers 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:20.751667 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a598d6cbb84578ab82e3807b98f36f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
I20240501 07:59:20.751754 2671 raft_consensus.cc:3037] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.751811 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a598d6cbb84578ab82e3807b98f36f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691"
I20240501 07:59:20.751827 2800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a598d6cbb84578ab82e3807b98f36f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:20.751909 2408 raft_consensus.cc:3037] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.751950 2800 raft_consensus.cc:3037] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.752080 2539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97a598d6cbb84578ab82e3807b98f36f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:20.752169 2539 raft_consensus.cc:3037] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.752740 2671 raft_consensus.cc:2454] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:20.752874 2408 raft_consensus.cc:2454] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:20.752985 2539 raft_consensus.cc:2454] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:20.753186 2800 raft_consensus.cc:2454] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:20.753563 2868 leader_election.cc:302] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 5dd2fd1701284a6b886ab8d14d1bd8df, 85377a976c05408fbb384237a0edb691, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:20.753808 3504 raft_consensus.cc:2781] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:20.753895 3504 raft_consensus.cc:686] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Becoming Leader. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Running, Role: LEADER
I20240501 07:59:20.754073 3466 raft_consensus.cc:482] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:20.754069 3504 consensus_queue.cc:237] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } }
I20240501 07:59:20.754159 3466 raft_consensus.cc:504] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.754698 3466 leader_election.cc:288] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829)
I20240501 07:59:20.755100 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382e9b7922f849f980b80ded29e99075" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0" is_pre_election: true
I20240501 07:59:20.755225 3064 raft_consensus.cc:2454] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:20.755290 3466 raft_consensus.cc:482] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:20.755362 3466 raft_consensus.cc:504] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.755713 2800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382e9b7922f849f980b80ded29e99075" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:20.755786 3466 leader_election.cc:288] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027)
I20240501 07:59:20.755820 2800 raft_consensus.cc:2454] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:20.756083 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ade42f3f20e4deda01e89ee6ee4141f" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0" is_pre_election: true
I20240501 07:59:20.755985 2263 catalog_manager.cc:5484] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:20.756126 2606 leader_election.cc:302] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 3bd6898f790b48d593253180b79c5e40, 5dd2fd1701284a6b886ab8d14d1bd8df; no voters:
I20240501 07:59:20.756179 3064 raft_consensus.cc:2454] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:20.756332 2409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ade42f3f20e4deda01e89ee6ee4141f" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691" is_pre_election: true
I20240501 07:59:20.756456 2409 raft_consensus.cc:2454] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:20.756678 2932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ade42f3f20e4deda01e89ee6ee4141f" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" is_pre_election: true
I20240501 07:59:20.756716 2606 leader_election.cc:302] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 5dd2fd1701284a6b886ab8d14d1bd8df, 85377a976c05408fbb384237a0edb691; no voters:
I20240501 07:59:20.756803 2932 raft_consensus.cc:2454] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:20.756831 3466 raft_consensus.cc:2781] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:20.756875 3466 raft_consensus.cc:482] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:20.756904 3466 raft_consensus.cc:3037] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.757637 3466 raft_consensus.cc:504] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.758070 3466 leader_election.cc:288] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 election: Requested vote from peers 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829)
I20240501 07:59:20.758165 3466 raft_consensus.cc:2781] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:20.758212 3466 raft_consensus.cc:482] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:20.758255 3466 raft_consensus.cc:3037] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.758402 2800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382e9b7922f849f980b80ded29e99075" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:20.758497 2800 raft_consensus.cc:3037] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.758972 3466 raft_consensus.cc:504] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.759261 2800 raft_consensus.cc:2454] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:20.759416 3466 leader_election.cc:288] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 election: Requested vote from peers 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027)
I20240501 07:59:20.759721 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382e9b7922f849f980b80ded29e99075" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691" is_pre_election: true
I20240501 07:59:20.759826 2408 raft_consensus.cc:2454] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:20.759900 3063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ade42f3f20e4deda01e89ee6ee4141f" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0"
I20240501 07:59:20.759981 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382e9b7922f849f980b80ded29e99075" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0"
I20240501 07:59:20.760061 3063 raft_consensus.cc:3037] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.760076 3064 raft_consensus.cc:3037] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.760242 2409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ade42f3f20e4deda01e89ee6ee4141f" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691"
I20240501 07:59:20.760320 2409 raft_consensus.cc:3037] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.761060 3064 raft_consensus.cc:2454] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:20.761093 2409 raft_consensus.cc:2454] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:20.761404 2607 leader_election.cc:302] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 3bd6898f790b48d593253180b79c5e40, 5dd2fd1701284a6b886ab8d14d1bd8df; no voters:
I20240501 07:59:20.761438 3063 raft_consensus.cc:2454] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:20.761428 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382e9b7922f849f980b80ded29e99075" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691"
I20240501 07:59:20.761523 2408 raft_consensus.cc:3037] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.761564 3332 raft_consensus.cc:2781] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:20.761636 3332 raft_consensus.cc:686] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Becoming Leader. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Running, Role: LEADER
I20240501 07:59:20.761701 2932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ade42f3f20e4deda01e89ee6ee4141f" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2"
I20240501 07:59:20.761763 3332 consensus_queue.cc:237] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.761850 2932 raft_consensus.cc:3037] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.762311 2408 raft_consensus.cc:2454] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:20.762718 2932 raft_consensus.cc:2454] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:20.763008 2606 leader_election.cc:302] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 5dd2fd1701284a6b886ab8d14d1bd8df, 85377a976c05408fbb384237a0edb691, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:20.763151 3332 raft_consensus.cc:2781] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:20.763232 3332 raft_consensus.cc:686] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Becoming Leader. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Running, Role: LEADER
I20240501 07:59:20.763355 3332 consensus_queue.cc:237] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } }
I20240501 07:59:20.763367 2538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382e9b7922f849f980b80ded29e99075" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:20.763415 2540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ade42f3f20e4deda01e89ee6ee4141f" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:20.763478 2538 raft_consensus.cc:3037] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.763525 2540 raft_consensus.cc:2454] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:20.763372 2539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382e9b7922f849f980b80ded29e99075" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:20.763734 2263 catalog_manager.cc:5484] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: term changed from 0 to 1, leader changed from <none> to 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131). New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:20.764087 2537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ade42f3f20e4deda01e89ee6ee4141f" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:20.764214 2537 raft_consensus.cc:3037] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:20.764395 2538 raft_consensus.cc:2454] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:20.765000 2537 raft_consensus.cc:2454] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:20.767031 2263 catalog_manager.cc:5484] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: term changed from 0 to 1, leader changed from <none> to 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131). New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } }
W20240501 07:59:20.871376 3580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:20.871595 3580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:20.874563 3580 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240501 07:59:20.972507 2537 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.972556 2408 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.972955 2800 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.973526 3504 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.973747 3505 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.974089 2671 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.977154 3505 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.978457 2801 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.978962 3063 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.978964 2538 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.979722 3332 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.981911 2408 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:20.981875 3505 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.982124 3544 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.982383 3544 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:20.992439 3332 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.019454 2408 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.021970 3064 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.022128 3466 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.022429 3466 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.024101 2932 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 07:59:21.026254 2537 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 07:59:21.026638 3332 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.031369 3332 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 165.142 ms
time per row: 0.082571 ms
I20240501 07:59:21.088373 2262 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35940:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:21.093152 2782 tablet_service.cc:1467] Processing CreateTablet for tablet 6949f70a1ae44576b42d388231f0b50f (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.093281 2913 tablet_service.cc:1467] Processing CreateTablet for tablet 6949f70a1ae44576b42d388231f0b50f (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.093451 2782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6949f70a1ae44576b42d388231f0b50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.093569 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6949f70a1ae44576b42d388231f0b50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.093601 2519 tablet_service.cc:1467] Processing CreateTablet for tablet 6949f70a1ae44576b42d388231f0b50f (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.093820 2519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6949f70a1ae44576b42d388231f0b50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.095408 2650 tablet_service.cc:1467] Processing CreateTablet for tablet a42ea1bbca294f3e9298d895eee087c7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.095605 3555 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:21.095604 3557 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:21.095665 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42ea1bbca294f3e9298d895eee087c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.096024 2388 tablet_service.cc:1467] Processing CreateTablet for tablet 6949f70a1ae44576b42d388231f0b50f (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.096269 2388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6949f70a1ae44576b42d388231f0b50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.096379 3557 tablet_bootstrap.cc:654] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.096426 3555 tablet_bootstrap.cc:654] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.097481 3555 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:21.097535 3556 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:21.097555 3555 ts_tablet_manager.cc:1367] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:21.097966 3555 raft_consensus.cc:348] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.098074 3555 raft_consensus.cc:374] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.098107 3555 raft_consensus.cc:729] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.098229 3555 consensus_queue.cc:260] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.098263 2912 tablet_service.cc:1467] Processing CreateTablet for tablet f64014c3e7474e21b65082390fc1e6a6 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.098325 3556 tablet_bootstrap.cc:654] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.098482 2912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64014c3e7474e21b65082390fc1e6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.098488 3555 ts_tablet_manager.cc:1398] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.098429 2389 tablet_service.cc:1467] Processing CreateTablet for tablet f64014c3e7474e21b65082390fc1e6a6 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.098659 2389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64014c3e7474e21b65082390fc1e6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.099294 3556 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:21.099368 3556 ts_tablet_manager.cc:1367] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:21.099768 3556 raft_consensus.cc:348] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.099865 3556 raft_consensus.cc:374] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.099891 3556 raft_consensus.cc:729] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.100095 3556 consensus_queue.cc:260] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.100378 3556 ts_tablet_manager.cc:1398] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.100533 3561 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:21.101148 2519 tablet_service.cc:1467] Processing CreateTablet for tablet a42ea1bbca294f3e9298d895eee087c7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.101228 3561 tablet_bootstrap.cc:654] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.101361 2519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42ea1bbca294f3e9298d895eee087c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.101723 2913 tablet_service.cc:1467] Processing CreateTablet for tablet a42ea1bbca294f3e9298d895eee087c7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.101915 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42ea1bbca294f3e9298d895eee087c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.102180 3558 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:21.102982 3558 tablet_bootstrap.cc:654] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.103238 2518 tablet_service.cc:1467] Processing CreateTablet for tablet f64014c3e7474e21b65082390fc1e6a6 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.103438 2518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64014c3e7474e21b65082390fc1e6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.104055 3042 tablet_service.cc:1467] Processing CreateTablet for tablet 6949f70a1ae44576b42d388231f0b50f (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.104305 3042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6949f70a1ae44576b42d388231f0b50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.105093 3043 tablet_service.cc:1467] Processing CreateTablet for tablet f64014c3e7474e21b65082390fc1e6a6 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.105304 3043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64014c3e7474e21b65082390fc1e6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.106022 3560 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:21.106660 3560 tablet_bootstrap.cc:654] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.106837 3044 tablet_service.cc:1467] Processing CreateTablet for tablet a42ea1bbca294f3e9298d895eee087c7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.107017 3044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42ea1bbca294f3e9298d895eee087c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.107496 3560 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:21.107551 3560 ts_tablet_manager.cc:1367] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:21.108145 3560 raft_consensus.cc:348] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.108246 3560 raft_consensus.cc:374] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.108278 3560 raft_consensus.cc:729] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.108384 3560 consensus_queue.cc:260] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.108640 3560 ts_tablet_manager.cc:1398] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.108812 3560 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:21.108758 2782 tablet_service.cc:1467] Processing CreateTablet for tablet a42ea1bbca294f3e9298d895eee087c7 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.108886 3558 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:21.108958 3558 ts_tablet_manager.cc:1367] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240501 07:59:21.109054 2782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42ea1bbca294f3e9298d895eee087c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.109369 3558 raft_consensus.cc:348] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.109489 3558 raft_consensus.cc:374] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.109522 3558 raft_consensus.cc:729] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.109591 3560 tablet_bootstrap.cc:654] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.109647 3558 consensus_queue.cc:260] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.108758 2780 tablet_service.cc:1467] Processing CreateTablet for tablet f64014c3e7474e21b65082390fc1e6a6 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.109918 3558 ts_tablet_manager.cc:1398] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:21.110038 2780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64014c3e7474e21b65082390fc1e6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.110095 3558 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:21.110903 3558 tablet_bootstrap.cc:654] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.111713 3555 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:21.111742 3561 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:21.111820 3561 ts_tablet_manager.cc:1367] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240501 07:59:21.112270 3561 raft_consensus.cc:348] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.112380 3561 raft_consensus.cc:374] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.112412 3561 raft_consensus.cc:729] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.112536 3561 consensus_queue.cc:260] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.112777 3555 tablet_bootstrap.cc:654] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.112793 3561 ts_tablet_manager.cc:1398] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.112954 3561 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:21.113772 3555 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:21.113786 3561 tablet_bootstrap.cc:654] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.113848 3555 ts_tablet_manager.cc:1367] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:21.114292 3555 raft_consensus.cc:348] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.114421 3555 raft_consensus.cc:374] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.114452 3555 raft_consensus.cc:729] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.114557 3555 consensus_queue.cc:260] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.114707 3561 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:21.114784 3561 ts_tablet_manager.cc:1367] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:21.114802 3555 ts_tablet_manager.cc:1398] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.114934 3555 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:21.115160 3561 raft_consensus.cc:348] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.115250 3561 raft_consensus.cc:374] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.115283 3561 raft_consensus.cc:729] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.115396 3561 consensus_queue.cc:260] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.115710 3555 tablet_bootstrap.cc:654] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.116148 3561 ts_tablet_manager.cc:1398] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.116298 3561 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:21.117110 3561 tablet_bootstrap.cc:654] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.117663 3555 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:21.117734 3555 ts_tablet_manager.cc:1367] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:21.118132 3560 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:21.118135 3555 raft_consensus.cc:348] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.118224 3560 ts_tablet_manager.cc:1367] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240501 07:59:21.118238 3555 raft_consensus.cc:374] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.118268 3555 raft_consensus.cc:729] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.118381 3555 consensus_queue.cc:260] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.118615 3555 ts_tablet_manager.cc:1398] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.118634 3560 raft_consensus.cc:348] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.118729 3560 raft_consensus.cc:374] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.118753 3560 raft_consensus.cc:729] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.118827 3558 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:21.118850 3560 consensus_queue.cc:260] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.118903 3558 ts_tablet_manager.cc:1367] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240501 07:59:21.119086 3560 ts_tablet_manager.cc:1398] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.119228 3560 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:21.119297 3558 raft_consensus.cc:348] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.119386 3558 raft_consensus.cc:374] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.119405 3558 raft_consensus.cc:729] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.119498 3558 consensus_queue.cc:260] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.119742 3558 ts_tablet_manager.cc:1398] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:21.120405 3557 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:21.120507 3557 ts_tablet_manager.cc:1367] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20240501 07:59:21.120947 3557 raft_consensus.cc:348] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.121057 3557 raft_consensus.cc:374] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.121078 3557 raft_consensus.cc:729] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.121179 3557 consensus_queue.cc:260] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.121415 3557 ts_tablet_manager.cc:1398] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.121556 3557 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:21.122300 3557 tablet_bootstrap.cc:654] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.123306 3557 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:21.123354 3560 tablet_bootstrap.cc:654] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.123461 3557 ts_tablet_manager.cc:1367] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:21.124056 3557 raft_consensus.cc:348] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.124215 3557 raft_consensus.cc:374] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.124290 3557 raft_consensus.cc:729] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.124430 3561 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:21.124516 3561 ts_tablet_manager.cc:1367] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240501 07:59:21.124511 3557 consensus_queue.cc:260] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.124930 3557 ts_tablet_manager.cc:1398] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.124920 3561 raft_consensus.cc:348] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.125042 3561 raft_consensus.cc:374] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.125087 3561 raft_consensus.cc:729] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.125221 3557 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:21.125205 3561 consensus_queue.cc:260] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.125449 3561 ts_tablet_manager.cc:1398] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.126035 3557 tablet_bootstrap.cc:654] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.126186 3560 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:21.126263 3560 ts_tablet_manager.cc:1367] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240501 07:59:21.126611 3560 raft_consensus.cc:348] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.126730 3560 raft_consensus.cc:374] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.126770 3560 raft_consensus.cc:729] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.126889 3560 consensus_queue.cc:260] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.126987 3557 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:21.127054 3557 ts_tablet_manager.cc:1367] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:21.127161 3560 ts_tablet_manager.cc:1398] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:21.127418 3557 raft_consensus.cc:348] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.127516 3557 raft_consensus.cc:374] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.127548 3557 raft_consensus.cc:729] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.127660 3557 consensus_queue.cc:260] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.127912 3557 ts_tablet_manager.cc:1398] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.128962 3504 raft_consensus.cc:482] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:21.129045 3504 raft_consensus.cc:504] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.129524 3504 leader_election.cc:288] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933)
I20240501 07:59:21.129684 2801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:21.129799 2801 raft_consensus.cc:2454] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:21.129938 2537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:21.130031 2537 raft_consensus.cc:2454] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:21.130064 2409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691" is_pre_election: true
I20240501 07:59:21.130184 2409 raft_consensus.cc:2454] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:21.130249 3063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0" is_pre_election: true
I20240501 07:59:21.130344 3063 raft_consensus.cc:2454] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:21.130451 2868 leader_election.cc:302] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 5810fd56ce9244438edcd3c0ea23e72c, 85377a976c05408fbb384237a0edb691, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:21.130599 3504 raft_consensus.cc:2781] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:21.130650 3504 raft_consensus.cc:482] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:21.130703 3504 raft_consensus.cc:3037] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.131500 3504 raft_consensus.cc:504] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.131877 3504 leader_election.cc:288] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 election: Requested vote from peers 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933)
I20240501 07:59:21.132112 2537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:21.132135 3063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0"
I20240501 07:59:21.132198 2537 raft_consensus.cc:3037] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.132205 3063 raft_consensus.cc:3037] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.132231 2801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:21.132246 2409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691"
I20240501 07:59:21.132324 2409 raft_consensus.cc:3037] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.132342 2801 raft_consensus.cc:3037] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.133038 3063 raft_consensus.cc:2454] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:21.133040 2537 raft_consensus.cc:2454] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:21.133200 2409 raft_consensus.cc:2454] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:21.133200 2801 raft_consensus.cc:2454] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:21.133630 2868 leader_election.cc:302] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 85377a976c05408fbb384237a0edb691, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:21.133785 3504 raft_consensus.cc:2781] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:21.133881 3504 raft_consensus.cc:686] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Becoming Leader. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Running, Role: LEADER
I20240501 07:59:21.133999 3504 consensus_queue.cc:237] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.135458 2263 catalog_manager.cc:5484] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:21.172199 3504 raft_consensus.cc:482] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:21.172312 3504 raft_consensus.cc:504] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.172814 3504 leader_election.cc:288] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933)
I20240501 07:59:21.173059 2801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64014c3e7474e21b65082390fc1e6a6" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:21.173107 2537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64014c3e7474e21b65082390fc1e6a6" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:21.173102 3063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64014c3e7474e21b65082390fc1e6a6" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0" is_pre_election: true
I20240501 07:59:21.173192 2801 raft_consensus.cc:2454] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:21.173236 2537 raft_consensus.cc:2454] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:21.173236 3063 raft_consensus.cc:2454] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:21.173420 2409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64014c3e7474e21b65082390fc1e6a6" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691" is_pre_election: true
I20240501 07:59:21.173560 2409 raft_consensus.cc:2454] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:21.173554 2869 leader_election.cc:302] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 4 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 3bd6898f790b48d593253180b79c5e40, 5810fd56ce9244438edcd3c0ea23e72c, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:21.173763 3504 raft_consensus.cc:2781] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:21.173822 3504 raft_consensus.cc:482] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:21.173859 3504 raft_consensus.cc:3037] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.174618 3504 raft_consensus.cc:504] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.174984 3504 leader_election.cc:288] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 election: Requested vote from peers 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933)
I20240501 07:59:21.175200 2801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64014c3e7474e21b65082390fc1e6a6" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:21.175228 2537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64014c3e7474e21b65082390fc1e6a6" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:21.175305 2801 raft_consensus.cc:3037] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.175318 2537 raft_consensus.cc:3037] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.175287 2409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64014c3e7474e21b65082390fc1e6a6" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691"
I20240501 07:59:21.175383 2409 raft_consensus.cc:3037] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.175392 3063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64014c3e7474e21b65082390fc1e6a6" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0"
I20240501 07:59:21.175477 3063 raft_consensus.cc:3037] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.176172 2537 raft_consensus.cc:2454] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:21.176172 2801 raft_consensus.cc:2454] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:21.176276 3063 raft_consensus.cc:2454] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:21.176276 2409 raft_consensus.cc:2454] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:21.176502 2869 leader_election.cc:302] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 5810fd56ce9244438edcd3c0ea23e72c, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:21.176649 3504 raft_consensus.cc:2781] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:21.176729 3504 raft_consensus.cc:686] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Becoming Leader. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Running, Role: LEADER
I20240501 07:59:21.177028 3504 consensus_queue.cc:237] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } }
I20240501 07:59:21.179008 2263 catalog_manager.cc:5484] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:21.181708 3504 raft_consensus.cc:482] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:21.181782 3504 raft_consensus.cc:504] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.182384 2801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42ea1bbca294f3e9298d895eee087c7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:21.182404 2537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42ea1bbca294f3e9298d895eee087c7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:21.182385 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42ea1bbca294f3e9298d895eee087c7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" is_pre_election: true
I20240501 07:59:21.182510 2801 raft_consensus.cc:2454] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:21.182513 2670 raft_consensus.cc:2454] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:21.182518 2537 raft_consensus.cc:2454] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:21.182718 3504 leader_election.cc:288] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027)
I20240501 07:59:21.182833 2866 leader_election.cc:302] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 5810fd56ce9244438edcd3c0ea23e72c, 5dd2fd1701284a6b886ab8d14d1bd8df, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:21.183030 3063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42ea1bbca294f3e9298d895eee087c7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0" is_pre_election: true
I20240501 07:59:21.183127 3063 raft_consensus.cc:2454] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 0.
I20240501 07:59:21.183257 3504 raft_consensus.cc:2781] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:21.183310 3504 raft_consensus.cc:482] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:21.183359 3504 raft_consensus.cc:3037] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.184031 3504 raft_consensus.cc:504] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.184427 3504 leader_election.cc:288] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 election: Requested vote from peers 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027)
I20240501 07:59:21.184592 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42ea1bbca294f3e9298d895eee087c7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
I20240501 07:59:21.184628 2801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42ea1bbca294f3e9298d895eee087c7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:21.184715 2670 raft_consensus.cc:3037] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.184737 2801 raft_consensus.cc:3037] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.185359 2801 raft_consensus.cc:2454] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:21.185462 2670 raft_consensus.cc:2454] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:21.185639 3063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42ea1bbca294f3e9298d895eee087c7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0"
I20240501 07:59:21.185721 2867 leader_election.cc:302] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 3bd6898f790b48d593253180b79c5e40, 5dd2fd1701284a6b886ab8d14d1bd8df, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:21.185748 3063 raft_consensus.cc:3037] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.185895 2537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42ea1bbca294f3e9298d895eee087c7" candidate_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:21.185995 2537 raft_consensus.cc:3037] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.186439 3063 raft_consensus.cc:2454] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:21.186620 2537 raft_consensus.cc:2454] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2c508eb3dac4193a1eeba799e5d54f2 in term 1.
I20240501 07:59:21.186645 3504 raft_consensus.cc:2781] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:21.186720 3504 raft_consensus.cc:686] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Becoming Leader. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Running, Role: LEADER
I20240501 07:59:21.186841 3504 consensus_queue.cc:237] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } }
I20240501 07:59:21.188490 2263 catalog_manager.cc:5484] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:21.295679 3640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:21.296033 3640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:21.299060 3640 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240501 07:59:21.381431 2800 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.382004 3063 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.382388 2537 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.382753 2670 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.383234 3459 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.383374 3504 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.383623 2409 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.384181 2800 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.384198 3304 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.384303 3459 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.384490 2537 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.389591 3505 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.390050 3505 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.390091 3064 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.390187 3063 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.390288 3505 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.390566 2408 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.391131 3505 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.395519 2801 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.396531 2537 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.396911 3664 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.397168 3664 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.397258 3505 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.397331 3459 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 149.636 ms
time per row: 0.0748182 ms
I20240501 07:59:21.491663 2262 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35940:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:21.496753 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 5414588c25d048ed84ec9aae6f300ddf (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.496809 2782 tablet_service.cc:1467] Processing CreateTablet for tablet 5414588c25d048ed84ec9aae6f300ddf (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.496799 2389 tablet_service.cc:1467] Processing CreateTablet for tablet 5414588c25d048ed84ec9aae6f300ddf (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.497045 2782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5414588c25d048ed84ec9aae6f300ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.497047 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5414588c25d048ed84ec9aae6f300ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.497073 2389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5414588c25d048ed84ec9aae6f300ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.497762 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 2a62d5460d7844ebb51f4b33f915bd74 (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.497987 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a62d5460d7844ebb51f4b33f915bd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.498435 2649 tablet_service.cc:1467] Processing CreateTablet for tablet ceea369d9dcb4a8c94ec88adde948ef6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.498646 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceea369d9dcb4a8c94ec88adde948ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.499222 3555 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:21.500079 3555 tablet_bootstrap.cc:654] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.500361 3556 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:21.500972 3556 tablet_bootstrap.cc:654] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.500983 3555 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:21.501049 3555 ts_tablet_manager.cc:1367] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:21.501410 3555 raft_consensus.cc:348] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.501540 3555 raft_consensus.cc:374] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.501579 3555 raft_consensus.cc:729] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.501698 3555 consensus_queue.cc:260] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.501758 2518 tablet_service.cc:1467] Processing CreateTablet for tablet 5414588c25d048ed84ec9aae6f300ddf (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.501868 3556 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:21.501933 3556 ts_tablet_manager.cc:1367] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:21.501961 3555 ts_tablet_manager.cc:1398] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.502005 2518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5414588c25d048ed84ec9aae6f300ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.502336 3556 raft_consensus.cc:348] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.502431 3556 raft_consensus.cc:374] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.502462 3556 raft_consensus.cc:729] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.502547 2782 tablet_service.cc:1467] Processing CreateTablet for tablet 2a62d5460d7844ebb51f4b33f915bd74 (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.502575 3556 consensus_queue.cc:260] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.502743 2782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a62d5460d7844ebb51f4b33f915bd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.502828 3556 ts_tablet_manager.cc:1398] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.502970 3556 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:21.504180 2387 tablet_service.cc:1467] Processing CreateTablet for tablet ceea369d9dcb4a8c94ec88adde948ef6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.504217 2519 tablet_service.cc:1467] Processing CreateTablet for tablet 2a62d5460d7844ebb51f4b33f915bd74 (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.504285 3557 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:21.504426 2519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a62d5460d7844ebb51f4b33f915bd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.504429 2387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceea369d9dcb4a8c94ec88adde948ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.504735 3044 tablet_service.cc:1467] Processing CreateTablet for tablet 2a62d5460d7844ebb51f4b33f915bd74 (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.504174 2388 tablet_service.cc:1467] Processing CreateTablet for tablet 2a62d5460d7844ebb51f4b33f915bd74 (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.504968 3044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a62d5460d7844ebb51f4b33f915bd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.505054 2388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a62d5460d7844ebb51f4b33f915bd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.506165 2520 tablet_service.cc:1467] Processing CreateTablet for tablet ceea369d9dcb4a8c94ec88adde948ef6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.506374 2520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceea369d9dcb4a8c94ec88adde948ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.506757 3560 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:21.506925 3043 tablet_service.cc:1467] Processing CreateTablet for tablet ceea369d9dcb4a8c94ec88adde948ef6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.507120 3043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceea369d9dcb4a8c94ec88adde948ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.507592 3560 tablet_bootstrap.cc:654] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.508435 3556 tablet_bootstrap.cc:654] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.509034 2913 tablet_service.cc:1467] Processing CreateTablet for tablet 5414588c25d048ed84ec9aae6f300ddf (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.509279 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5414588c25d048ed84ec9aae6f300ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.509320 2912 tablet_service.cc:1467] Processing CreateTablet for tablet ceea369d9dcb4a8c94ec88adde948ef6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:21.509534 2912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceea369d9dcb4a8c94ec88adde948ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:21.510849 3561 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:21.511570 3555 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40: Bootstrap starting.
I20240501 07:59:21.511608 3561 tablet_bootstrap.cc:654] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.512476 3555 tablet_bootstrap.cc:654] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.512552 3561 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:21.512632 3561 ts_tablet_manager.cc:1367] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:21.513024 3561 raft_consensus.cc:348] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.513136 3561 raft_consensus.cc:374] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.513172 3561 raft_consensus.cc:729] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.513279 3561 consensus_queue.cc:260] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.513546 3561 ts_tablet_manager.cc:1398] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.513656 3555 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40: No bootstrap required, opened a new log
I20240501 07:59:21.513696 3561 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2: Bootstrap starting.
I20240501 07:59:21.513756 3555 ts_tablet_manager.cc:1367] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 07:59:21.514165 3555 raft_consensus.cc:348] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.514288 3555 raft_consensus.cc:374] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.514329 3555 raft_consensus.cc:729] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.514456 3555 consensus_queue.cc:260] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.514534 3561 tablet_bootstrap.cc:654] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.514727 3555 ts_tablet_manager.cc:1398] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:21.515673 3561 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2: No bootstrap required, opened a new log
I20240501 07:59:21.515754 3561 ts_tablet_manager.cc:1367] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:21.516451 3561 raft_consensus.cc:348] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.516534 3561 raft_consensus.cc:374] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.516561 3561 raft_consensus.cc:729] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.516644 3561 consensus_queue.cc:260] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.516873 3561 ts_tablet_manager.cc:1398] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.524039 3557 tablet_bootstrap.cc:654] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.525847 3558 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:21.526410 3557 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:21.526520 3557 ts_tablet_manager.cc:1367] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.002s
I20240501 07:59:21.526705 3558 tablet_bootstrap.cc:654] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.526926 3557 raft_consensus.cc:348] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.527017 3557 raft_consensus.cc:374] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.527038 3557 raft_consensus.cc:729] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.527143 3557 consensus_queue.cc:260] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.527436 3557 ts_tablet_manager.cc:1398] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:21.527599 3557 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:21.527662 3558 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:21.527743 3558 ts_tablet_manager.cc:1367] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 07:59:21.528187 3558 raft_consensus.cc:348] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.528295 3558 raft_consensus.cc:374] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.528329 3558 raft_consensus.cc:729] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.528445 3558 consensus_queue.cc:260] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.528676 3558 ts_tablet_manager.cc:1398] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.528693 3557 tablet_bootstrap.cc:654] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.528843 3558 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:21.530277 3558 tablet_bootstrap.cc:654] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.530678 3557 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:21.530745 3557 ts_tablet_manager.cc:1367] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 07:59:21.531111 3557 raft_consensus.cc:348] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.531229 3557 raft_consensus.cc:374] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.531265 3557 raft_consensus.cc:729] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.531368 3557 consensus_queue.cc:260] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.531522 3558 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:21.531594 3558 ts_tablet_manager.cc:1367] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:21.531635 3557 ts_tablet_manager.cc:1398] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:21.531797 3557 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:21.532264 3560 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:21.532357 3560 ts_tablet_manager.cc:1367] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.002s
I20240501 07:59:21.532604 3557 tablet_bootstrap.cc:654] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.532733 3560 raft_consensus.cc:348] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.532825 3560 raft_consensus.cc:374] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.532858 3560 raft_consensus.cc:729] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.532971 3560 consensus_queue.cc:260] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.533231 3560 ts_tablet_manager.cc:1398] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:21.533385 3560 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0: Bootstrap starting.
I20240501 07:59:21.533555 3556 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:21.533636 3556 ts_tablet_manager.cc:1367] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.000s
I20240501 07:59:21.534040 3556 raft_consensus.cc:348] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.534152 3556 raft_consensus.cc:374] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.534183 3556 raft_consensus.cc:729] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.534294 3556 consensus_queue.cc:260] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.534389 3560 tablet_bootstrap.cc:654] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.534551 3556 ts_tablet_manager.cc:1398] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:21.534708 3556 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:21.535471 3560 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0: No bootstrap required, opened a new log
I20240501 07:59:21.535557 3560 ts_tablet_manager.cc:1367] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 07:59:21.535714 3556 tablet_bootstrap.cc:654] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.535909 3560 raft_consensus.cc:348] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.536087 3560 raft_consensus.cc:374] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.536128 3560 raft_consensus.cc:729] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.536252 3560 consensus_queue.cc:260] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.536549 3560 ts_tablet_manager.cc:1398] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:21.536895 3556 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df: No bootstrap required, opened a new log
I20240501 07:59:21.536967 3556 ts_tablet_manager.cc:1367] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:21.537377 3556 raft_consensus.cc:348] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.537479 3556 raft_consensus.cc:374] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.537513 3556 raft_consensus.cc:729] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.537598 3556 consensus_queue.cc:260] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.537691 3558 raft_consensus.cc:348] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.537827 3556 ts_tablet_manager.cc:1398] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:21.537781 3558 raft_consensus.cc:374] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.538151 3558 raft_consensus.cc:729] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.538285 3558 consensus_queue.cc:260] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.538555 3558 ts_tablet_manager.cc:1398] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240501 07:59:21.538884 3558 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691: Bootstrap starting.
I20240501 07:59:21.539947 3558 tablet_bootstrap.cc:654] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:21.540539 3557 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c: No bootstrap required, opened a new log
I20240501 07:59:21.540658 3557 ts_tablet_manager.cc:1367] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240501 07:59:21.541002 3558 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691: No bootstrap required, opened a new log
I20240501 07:59:21.541085 3558 ts_tablet_manager.cc:1367] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:21.541112 3557 raft_consensus.cc:348] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.541222 3557 raft_consensus.cc:374] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.541265 3557 raft_consensus.cc:729] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.541373 3557 consensus_queue.cc:260] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.541499 3558 raft_consensus.cc:348] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.541607 3558 raft_consensus.cc:374] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:21.541646 3557 ts_tablet_manager.cc:1398] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:21.541646 3558 raft_consensus.cc:729] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377a976c05408fbb384237a0edb691, State: Initialized, Role: FOLLOWER
I20240501 07:59:21.541764 3558 consensus_queue.cc:260] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.542016 3558 ts_tablet_manager.cc:1398] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:21.542435 3617 raft_consensus.cc:482] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:21.542522 3617 raft_consensus.cc:504] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.543087 3617 leader_election.cc:288] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:21.543493 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceea369d9dcb4a8c94ec88adde948ef6" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" is_pre_election: true
I20240501 07:59:21.543541 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceea369d9dcb4a8c94ec88adde948ef6" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691" is_pre_election: true
I20240501 07:59:21.543627 2670 raft_consensus.cc:2454] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 0.
I20240501 07:59:21.543640 2408 raft_consensus.cc:2454] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 0.
I20240501 07:59:21.543900 2999 leader_election.cc:302] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 5dd2fd1701284a6b886ab8d14d1bd8df, 85377a976c05408fbb384237a0edb691; no voters:
I20240501 07:59:21.544123 2538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceea369d9dcb4a8c94ec88adde948ef6" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:21.544235 2538 raft_consensus.cc:2454] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 0.
I20240501 07:59:21.544399 3617 raft_consensus.cc:2781] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:21.544452 3617 raft_consensus.cc:482] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:21.544492 3617 raft_consensus.cc:3037] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.545431 3617 raft_consensus.cc:504] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.545874 3617 leader_election.cc:288] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [CANDIDATE]: Term 1 election: Requested vote from peers e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:21.546077 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceea369d9dcb4a8c94ec88adde948ef6" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
I20240501 07:59:21.546171 2670 raft_consensus.cc:3037] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.546214 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceea369d9dcb4a8c94ec88adde948ef6" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691"
I20240501 07:59:21.546293 2408 raft_consensus.cc:3037] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.546293 2538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceea369d9dcb4a8c94ec88adde948ef6" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:21.546378 2538 raft_consensus.cc:3037] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.547056 2670 raft_consensus.cc:2454] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 1.
I20240501 07:59:21.547055 2408 raft_consensus.cc:2454] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 1.
I20240501 07:59:21.547252 2538 raft_consensus.cc:2454] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 1.
I20240501 07:59:21.547626 2997 leader_election.cc:302] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 5810fd56ce9244438edcd3c0ea23e72c, 5dd2fd1701284a6b886ab8d14d1bd8df; no voters:
I20240501 07:59:21.547797 3617 raft_consensus.cc:2781] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:21.548000 3617 raft_consensus.cc:686] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 LEADER]: Becoming Leader. State: Replica: 1537338fbeea47ee81f7a210cd6df2f0, State: Running, Role: LEADER
I20240501 07:59:21.548123 3617 consensus_queue.cc:237] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.548846 2932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceea369d9dcb4a8c94ec88adde948ef6" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" is_pre_election: true
I20240501 07:59:21.548959 2932 raft_consensus.cc:2454] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 0.
I20240501 07:59:21.549163 2933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ceea369d9dcb4a8c94ec88adde948ef6" candidate_uuid: "1537338fbeea47ee81f7a210cd6df2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2"
I20240501 07:59:21.549268 2933 raft_consensus.cc:3037] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.550009 2933 raft_consensus.cc:2454] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1537338fbeea47ee81f7a210cd6df2f0 in term 1.
I20240501 07:59:21.550045 2262 catalog_manager.cc:5484] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134). New cstate: current_term: 1 leader_uuid: "1537338fbeea47ee81f7a210cd6df2f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:21.610878 3544 raft_consensus.cc:482] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:21.611016 3544 raft_consensus.cc:504] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.611465 3544 leader_election.cc:288] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:21.611778 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5414588c25d048ed84ec9aae6f300ddf" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691" is_pre_election: true
I20240501 07:59:21.611907 2408 raft_consensus.cc:2454] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:21.611902 2802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5414588c25d048ed84ec9aae6f300ddf" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:21.612066 2802 raft_consensus.cc:2454] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:21.612028 2538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5414588c25d048ed84ec9aae6f300ddf" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:21.612151 2538 raft_consensus.cc:2454] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:21.612398 2933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5414588c25d048ed84ec9aae6f300ddf" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" is_pre_election: true
I20240501 07:59:21.612505 2604 leader_election.cc:302] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 5810fd56ce9244438edcd3c0ea23e72c, 5dd2fd1701284a6b886ab8d14d1bd8df, 85377a976c05408fbb384237a0edb691; no voters:
I20240501 07:59:21.612521 2933 raft_consensus.cc:2454] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 0.
I20240501 07:59:21.612668 3544 raft_consensus.cc:2781] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:21.612725 3544 raft_consensus.cc:482] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:21.612766 3544 raft_consensus.cc:3037] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.613719 3544 raft_consensus.cc:504] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.614099 3544 leader_election.cc:288] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 election: Requested vote from peers 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829), e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:21.614370 2538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5414588c25d048ed84ec9aae6f300ddf" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:21.614467 2538 raft_consensus.cc:3037] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.614477 2408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5414588c25d048ed84ec9aae6f300ddf" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377a976c05408fbb384237a0edb691"
I20240501 07:59:21.614491 2802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5414588c25d048ed84ec9aae6f300ddf" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:21.614575 2408 raft_consensus.cc:3037] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.614585 2802 raft_consensus.cc:3037] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.615137 2538 raft_consensus.cc:2454] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:21.615448 2408 raft_consensus.cc:2454] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:21.615448 2802 raft_consensus.cc:2454] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:21.615486 2933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5414588c25d048ed84ec9aae6f300ddf" candidate_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2"
I20240501 07:59:21.615594 2933 raft_consensus.cc:3037] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.615763 2606 leader_election.cc:302] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 3bd6898f790b48d593253180b79c5e40, 5dd2fd1701284a6b886ab8d14d1bd8df, 85377a976c05408fbb384237a0edb691; no voters:
I20240501 07:59:21.616153 3544 raft_consensus.cc:2781] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:21.616238 3544 raft_consensus.cc:686] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Becoming Leader. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Running, Role: LEADER
I20240501 07:59:21.616366 3544 consensus_queue.cc:237] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } }
I20240501 07:59:21.616499 2933 raft_consensus.cc:2454] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dd2fd1701284a6b886ab8d14d1bd8df in term 1.
I20240501 07:59:21.618038 2262 catalog_manager.cc:5484] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: term changed from 0 to 1, leader changed from <none> to 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131). New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:21.737249 3553 raft_consensus.cc:482] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:21.737356 3553 raft_consensus.cc:504] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.737867 3553 leader_election.cc:288] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829)
I20240501 07:59:21.738186 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a62d5460d7844ebb51f4b33f915bd74" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0" is_pre_election: true
I20240501 07:59:21.738238 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a62d5460d7844ebb51f4b33f915bd74" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" is_pre_election: true
I20240501 07:59:21.738242 2538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a62d5460d7844ebb51f4b33f915bd74" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c" is_pre_election: true
I20240501 07:59:21.738322 3064 raft_consensus.cc:2454] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 0.
I20240501 07:59:21.738363 2670 raft_consensus.cc:2454] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 0.
I20240501 07:59:21.738361 2538 raft_consensus.cc:2454] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 0.
I20240501 07:59:21.738641 2342 leader_election.cc:302] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 1537338fbeea47ee81f7a210cd6df2f0, 5810fd56ce9244438edcd3c0ea23e72c, 85377a976c05408fbb384237a0edb691; no voters:
I20240501 07:59:21.738807 3553 raft_consensus.cc:2781] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:21.738860 3553 raft_consensus.cc:482] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:21.738894 3553 raft_consensus.cc:3037] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.739099 2802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a62d5460d7844ebb51f4b33f915bd74" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40" is_pre_election: true
I20240501 07:59:21.739226 2802 raft_consensus.cc:2454] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 0.
I20240501 07:59:21.739856 3553 raft_consensus.cc:504] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.740295 3553 leader_election.cc:288] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [CANDIDATE]: Term 1 election: Requested vote from peers 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829)
I20240501 07:59:21.740501 3064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a62d5460d7844ebb51f4b33f915bd74" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0"
I20240501 07:59:21.740523 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a62d5460d7844ebb51f4b33f915bd74" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
I20240501 07:59:21.740595 3064 raft_consensus.cc:3037] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.740613 2670 raft_consensus.cc:3037] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.740644 2538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a62d5460d7844ebb51f4b33f915bd74" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:21.740726 2538 raft_consensus.cc:3037] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.741459 2538 raft_consensus.cc:2454] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 1.
I20240501 07:59:21.741459 2670 raft_consensus.cc:2454] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 1.
I20240501 07:59:21.741479 3064 raft_consensus.cc:2454] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 1.
I20240501 07:59:21.741727 2343 leader_election.cc:302] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 5810fd56ce9244438edcd3c0ea23e72c, 5dd2fd1701284a6b886ab8d14d1bd8df, 85377a976c05408fbb384237a0edb691; no voters:
I20240501 07:59:21.741880 3553 raft_consensus.cc:2781] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:21.742028 3553 raft_consensus.cc:686] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Becoming Leader. State: Replica: 85377a976c05408fbb384237a0edb691, State: Running, Role: LEADER
I20240501 07:59:21.742136 2802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a62d5460d7844ebb51f4b33f915bd74" candidate_uuid: "85377a976c05408fbb384237a0edb691" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bd6898f790b48d593253180b79c5e40"
I20240501 07:59:21.742163 3553 consensus_queue.cc:237] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } }
I20240501 07:59:21.742235 2802 raft_consensus.cc:3037] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:21.743045 2802 raft_consensus.cc:2454] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85377a976c05408fbb384237a0edb691 in term 1.
I20240501 07:59:21.744028 2262 catalog_manager.cc:5484] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85377a976c05408fbb384237a0edb691 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:21.898458 3702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:21.898736 3702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:21.902866 3702 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240501 07:59:21.989571 2408 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.989567 2933 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.989614 2802 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.990132 2538 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.990635 3616 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.990787 3742 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.990880 3544 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.991107 3742 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.994567 2800 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.995260 3553 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:21.996358 2539 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:21.996729 3553 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:22.000243 3064 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:22.000304 2670 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:22.000630 3553 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:22.000929 3700 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:22.007077 2539 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:22.007583 2671 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:22.007583 2409 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:22.007704 2932 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:22.008138 3697 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:22.008728 3617 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:22.009069 3617 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:22.009325 3360 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:22.016887 3737 mvcc.cc:205] Tried to move back new op lower bound from 7022798282705190912 to 7022798281697951744. Current Snapshot: MvccSnapshot[applied={T|T < 7022798282705190912}]
I20240501 07:59:22.017455 3751 mvcc.cc:205] Tried to move back new op lower bound from 7022798282704556032 to 7022798281182580736. Current Snapshot: MvccSnapshot[applied={T|T < 7022798282704556032}]
I20240501 07:59:22.020710 3738 mvcc.cc:205] Tried to move back new op lower bound from 7022798282705190912 to 7022798281697951744. Current Snapshot: MvccSnapshot[applied={T|T < 7022798282705190912}]
I20240501 07:59:22.031611 3743 mvcc.cc:205] Tried to move back new op lower bound from 7022798282704556032 to 7022798281182580736. Current Snapshot: MvccSnapshot[applied={T|T < 7022798282704556032}]
I20240501 07:59:22.050627 3745 mvcc.cc:205] Tried to move back new op lower bound from 7022798282751029248 to 7022798280903553024. Current Snapshot: MvccSnapshot[applied={T|T < 7022798282751029248}]
I20240501 07:59:22.089041 3744 mvcc.cc:205] Tried to move back new op lower bound from 7022798282751029248 to 7022798280903553024. Current Snapshot: MvccSnapshot[applied={T|T < 7022798282751029248}]
INSERT report
rows total: 2000
time total: 166.479 ms
time per row: 0.0832395 ms
I20240501 07:59:22.111570 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.135:37389
--local_ip_for_outbound_sockets=127.0.99.135
--tserver_master_addrs=127.0.99.190:35233
--webserver_port=34939
--webserver_interface=127.0.99.135
--builtin_ntp_servers=127.0.99.148:36633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 07:59:22.213851 3788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:22.214047 3788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:22.214092 3788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:22.217043 3788 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:22.217100 3788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:22.217181 3788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.135
I20240501 07:59:22.221184 3788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:36633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.135:37389
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.0.99.135
--webserver_port=34939
--tserver_master_addrs=127.0.99.190:35233
--never_fsync=true
--heap_profile_path=/tmp/kudu.3788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.135
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:22.222100 3788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:22.223341 3788 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:22.232878 3796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:22.233498 3793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:22.234007 3794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:22.234622 3788 server_base.cc:1034] running on GCE node
I20240501 07:59:22.235023 3788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:22.235642 3788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:22.236799 3788 hybrid_clock.cc:648] HybridClock initialized: now 1714550362236742 us; error 89 us; skew 500 ppm
I20240501 07:59:22.238891 3788 webserver.cc:460] Webserver started at http://127.0.99.135:34939/ using document root <none> and password file <none>
I20240501 07:59:22.239600 3788 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:22.239696 3788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:22.243485 3788 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 07:59:22.245512 3802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:22.246709 3788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:22.246811 3788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "e48105f8f6ab476196dd66c4b12808d8"
format_stamp: "Formatted at 2024-05-01 07:59:15 on dist-test-slave-f81v"
I20240501 07:59:22.247287 3788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:22.277707 3788 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:22.278362 3788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:22.278503 3788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:22.279008 3788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:22.279840 3788 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:22.279886 3788 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:22.279954 3788 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:22.279982 3788 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:22.287827 3788 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.135:37389
I20240501 07:59:22.287855 3915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.135:37389 every 8 connection(s)
I20240501 07:59:22.288805 3788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 07:59:22.296722 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 3788
I20240501 07:59:22.296886 398 test_util.cc:268] Using random seed: -582266315
I20240501 07:59:22.299007 3916 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35233
I20240501 07:59:22.299222 3916 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:22.299790 3916 heartbeater.cc:511] Master 127.0.99.190:35233 requested a full tablet report, sending...
I20240501 07:59:22.300925 2258 ts_manager.cc:194] Re-registered known tserver with Master: e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389)
I20240501 07:59:22.301672 2258 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.135:55199
I20240501 07:59:22.307719 398 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_0 already exists
I20240501 07:59:22.307911 398 test_util.cc:268] Using random seed: -582255290
I20240501 07:59:22.316637 398 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_1 already exists
I20240501 07:59:22.316781 398 test_util.cc:268] Using random seed: -582246420
I20240501 07:59:22.327176 398 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_2 already exists
I20240501 07:59:22.327303 398 test_util.cc:268] Using random seed: -582235898
I20240501 07:59:22.341717 398 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_3 already exists
I20240501 07:59:22.341845 398 test_util.cc:268] Using random seed: -582221358
I20240501 07:59:22.358570 398 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_4 already exists
I20240501 07:59:22.903587 3043 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 07:59:22.943763 2782 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:22.964799 2912 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 07:59:22.985482 3850 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:22.996874 2649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 07:59:23.018757 2387 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:23.046170 2520 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:23.304322 3916 heartbeater.cc:503] Master 127.0.99.190:35233 was elected leader, sending a full tablet report...
W20240501 07:59:23.422760 2869 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.134:44027, user_credentials={real_user=slave}} blocked reactor thread for 46528.8us
I20240501 07:59:23.735605 2782 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:23.757763 3043 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 07:59:23.765514 2912 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 07:59:23.770968 3850 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:23.777678 2649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 07:59:23.944489 2387 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:23.973642 2520 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:24.220723 2933 consensus_queue.cc:237] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 36, Committed index: 36, Last appended: 1.36, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:24.224368 3059 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 36. Preceding OpId from leader: term: 1 index: 37. (index mismatch)
I20240501 07:59:24.224812 3995 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20240501 07:59:24.225513 2539 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 36. Preceding OpId from leader: term: 1 index: 37. (index mismatch)
I20240501 07:59:24.225672 2796 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 36. Preceding OpId from leader: term: 1 index: 37. (index mismatch)
I20240501 07:59:24.225844 3995 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20240501 07:59:24.226402 2407 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 36. Preceding OpId from leader: term: 1 index: 37. (index mismatch)
I20240501 07:59:24.227003 3995 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20240501 07:59:24.227254 3995 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 37, Last known committed idx: 36, Time since last communication: 0.000s
I20240501 07:59:24.231631 4003 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.37: config changed from index -1 to 37, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.235559 2258 catalog_manager.cc:5484] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: config changed from index -1 to 37, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:24.235672 2800 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.37: config changed from index -1 to 37, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.243542 2408 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.37: config changed from index -1 to 37, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.247429 2536 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.37: config changed from index -1 to 37, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
W20240501 07:59:24.263298 2868 consensus_peers.cc:485] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 -> Peer e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389): Couldn't send request to peer e48105f8f6ab476196dd66c4b12808d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f64014c3e7474e21b65082390fc1e6a6. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:24.335247 3063 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.37: config changed from index -1 to 37, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.414935 2409 consensus_queue.cc:237] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 1.27, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:24.419377 2670 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20240501 07:59:24.421427 3962 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20240501 07:59:24.426641 2933 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20240501 07:59:24.428017 3553 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
W20240501 07:59:24.430441 2344 consensus_peers.cc:485] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 -> Peer e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389): Couldn't send request to peer e48105f8f6ab476196dd66c4b12808d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1ae5c568fb854f2a9e4122fc21314024. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:24.430727 3553 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.431604 2799 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20240501 07:59:24.434211 2264 catalog_manager.cc:5484] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index -1 to 28, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:24.433995 2670 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.435432 3700 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20240501 07:59:24.436630 2799 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.442236 2931 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.452059 3062 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 27. Preceding OpId from leader: term: 1 index: 28. (index mismatch)
I20240501 07:59:24.481456 4067 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20240501 07:59:24.483047 3059 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.28: config changed from index -1 to 28, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.736436 2669 consensus_queue.cc:237] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 1.44, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:24.738325 2537 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240501 07:59:24.738664 4084 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 43, Time since last communication: 0.000s
I20240501 07:59:24.744081 4159 ts_tablet_manager.cc:902] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8: Initiating tablet copy from peer e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591)
I20240501 07:59:24.751080 2798 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240501 07:59:24.759728 2930 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240501 07:59:24.775224 4159 tablet_copy_client.cc:308] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:35591
I20240501 07:59:24.776644 4018 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 43, Time since last communication: 0.000s
I20240501 07:59:24.783526 4084 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 43, Time since last communication: 0.000s
I20240501 07:59:24.791236 2953 tablet_copy_service.cc:140] P e2c508eb3dac4193a1eeba799e5d54f2: Received BeginTabletCopySession request for tablet f64014c3e7474e21b65082390fc1e6a6 from peer e48105f8f6ab476196dd66c4b12808d8 ({username='slave'} at 127.0.99.135:50307)
I20240501 07:59:24.791396 2953 tablet_copy_service.cc:161] P e2c508eb3dac4193a1eeba799e5d54f2: Beginning new tablet copy session on tablet f64014c3e7474e21b65082390fc1e6a6 from peer e48105f8f6ab476196dd66c4b12808d8 at {username='slave'} at 127.0.99.135:50307: session id = e48105f8f6ab476196dd66c4b12808d8-f64014c3e7474e21b65082390fc1e6a6
I20240501 07:59:24.792865 2953 tablet_copy_source_session.cc:205] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:24.795245 4084 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.798239 4159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64014c3e7474e21b65082390fc1e6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:24.797730 2536 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.809753 2798 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.812760 2259 catalog_manager.cc:5484] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 reported cstate change: config changed from index -1 to 45, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.814535 2409 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240501 07:59:24.824038 4159 tablet_copy_client.cc:783] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:24.824225 4159 tablet_copy_client.cc:647] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 1 WAL segments...
W20240501 07:59:24.827980 2606 consensus_peers.cc:485] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df -> Peer e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389): Couldn't send request to peer e48105f8f6ab476196dd66c4b12808d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5414588c25d048ed84ec9aae6f300ddf. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:24.829233 4159 tablet_copy_client.cc:523] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:24.834167 2930 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.834594 4159 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap starting.
I20240501 07:59:24.853971 3616 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 43, Time since last communication: 0.000s
I20240501 07:59:24.858062 4159 log.cc:824] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:24.881116 2405 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:24.925753 4168 ts_tablet_manager.cc:902] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8: Initiating tablet copy from peer 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933)
I20240501 07:59:24.926324 4159 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=53 overwritten=0 applied=53 ignored=0} inserts{seen=725 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:24.935570 4159 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap complete.
I20240501 07:59:24.936012 4159 ts_tablet_manager.cc:1367] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8: Time spent bootstrapping tablet: real 0.102s user 0.062s sys 0.008s
I20240501 07:59:24.937595 4168 tablet_copy_client.cc:308] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:42933
I20240501 07:59:24.939518 4159 raft_consensus.cc:348] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:24.939776 4159 raft_consensus.cc:729] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e48105f8f6ab476196dd66c4b12808d8, State: Initialized, Role: LEARNER
I20240501 07:59:24.940156 4159 consensus_queue.cc:260] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 53, Last appended: 1.53, Last appended by leader: 53, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 37 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:24.941366 4159 ts_tablet_manager.cc:1398] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 07:59:24.955667 2953 tablet_copy_service.cc:342] P e2c508eb3dac4193a1eeba799e5d54f2: Request end of tablet copy session e48105f8f6ab476196dd66c4b12808d8-f64014c3e7474e21b65082390fc1e6a6 received from {username='slave'} at 127.0.99.135:50307
I20240501 07:59:24.955766 2953 tablet_copy_service.cc:433] P e2c508eb3dac4193a1eeba799e5d54f2: ending tablet copy session e48105f8f6ab476196dd66c4b12808d8-f64014c3e7474e21b65082390fc1e6a6 on tablet f64014c3e7474e21b65082390fc1e6a6 with peer e48105f8f6ab476196dd66c4b12808d8
I20240501 07:59:24.971812 2668 consensus_queue.cc:237] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 1.43, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: true } }
I20240501 07:59:24.974697 3059 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20240501 07:59:24.975198 4084 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
W20240501 07:59:24.976210 2604 consensus_peers.cc:485] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df -> Peer 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773): Couldn't send request to peer 5810fd56ce9244438edcd3c0ea23e72c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 490523aeb9e44f4ba8c2fc42fb73a95e. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:24.985476 2799 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20240501 07:59:24.985755 2930 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20240501 07:59:24.987453 2405 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20240501 07:59:24.991833 3740 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
I20240501 07:59:24.996212 3740 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
I20240501 07:59:25.003188 3740 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
I20240501 07:59:25.004421 4084 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: true } } }
I20240501 07:59:25.013684 2930 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: true } } }
I20240501 07:59:25.016705 2260 catalog_manager.cc:5484] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index -1 to 44, NON_VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:25.021010 3063 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: true } } }
I20240501 07:59:25.044893 2405 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: true } } }
I20240501 07:59:25.048439 2798 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: true } } }
I20240501 07:59:25.061024 2429 tablet_copy_service.cc:140] P 85377a976c05408fbb384237a0edb691: Received BeginTabletCopySession request for tablet 1ae5c568fb854f2a9e4122fc21314024 from peer e48105f8f6ab476196dd66c4b12808d8 ({username='slave'} at 127.0.99.135:52723)
I20240501 07:59:25.061172 2429 tablet_copy_service.cc:161] P 85377a976c05408fbb384237a0edb691: Beginning new tablet copy session on tablet 1ae5c568fb854f2a9e4122fc21314024 from peer e48105f8f6ab476196dd66c4b12808d8 at {username='slave'} at 127.0.99.135:52723: session id = e48105f8f6ab476196dd66c4b12808d8-1ae5c568fb854f2a9e4122fc21314024
I20240501 07:59:25.062587 2429 tablet_copy_source_session.cc:205] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:25.064252 4168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae5c568fb854f2a9e4122fc21314024. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:25.066406 4168 tablet_copy_client.cc:783] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:25.066551 4168 tablet_copy_client.cc:647] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:25.069017 4168 tablet_copy_client.cc:523] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:25.071204 4168 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap starting.
I20240501 07:59:25.091799 3870 raft_consensus.cc:1203] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.36->[1.37-1.60] Dedup: 1.53->[1.54-1.60]
I20240501 07:59:25.177106 2930 consensus_queue.cc:237] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 45, Committed index: 45, Last appended: 1.45, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: true } }
I20240501 07:59:25.182094 2537 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 45. Preceding OpId from leader: term: 1 index: 46. (index mismatch)
I20240501 07:59:25.182421 4159 ts_tablet_manager.cc:902] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: Initiating tablet copy from peer 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519)
I20240501 07:59:25.182473 3989 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 45, Time since last communication: 0.000s
W20240501 07:59:25.186326 2867 consensus_peers.cc:485] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 -> Peer 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519): Couldn't send request to peer 5dd2fd1701284a6b886ab8d14d1bd8df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6949f70a1ae44576b42d388231f0b50f. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:25.186892 4159 tablet_copy_client.cc:308] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:44519
W20240501 07:59:25.188678 2868 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.129:42933, user_credentials={real_user=slave}} blocked reactor thread for 50870us
I20240501 07:59:25.188860 3059 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 45. Preceding OpId from leader: term: 1 index: 46. (index mismatch)
I20240501 07:59:25.191594 2691 tablet_copy_service.cc:140] P 5dd2fd1701284a6b886ab8d14d1bd8df: Received BeginTabletCopySession request for tablet 5414588c25d048ed84ec9aae6f300ddf from peer e48105f8f6ab476196dd66c4b12808d8 ({username='slave'} at 127.0.99.135:36749)
I20240501 07:59:25.191731 2691 tablet_copy_service.cc:161] P 5dd2fd1701284a6b886ab8d14d1bd8df: Beginning new tablet copy session on tablet 5414588c25d048ed84ec9aae6f300ddf from peer e48105f8f6ab476196dd66c4b12808d8 at {username='slave'} at 127.0.99.135:36749: session id = e48105f8f6ab476196dd66c4b12808d8-5414588c25d048ed84ec9aae6f300ddf
I20240501 07:59:25.193207 2691 tablet_copy_source_session.cc:205] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:25.195964 4159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5414588c25d048ed84ec9aae6f300ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:25.193127 4157 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 45, Time since last communication: 0.000s
I20240501 07:59:25.203243 2798 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 45. Preceding OpId from leader: term: 1 index: 46. (index mismatch)
I20240501 07:59:25.204516 4159 tablet_copy_client.cc:783] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:25.204453 4158 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.46: config changed from index -1 to 46, NON_VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) added. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: true } } }
I20240501 07:59:25.204715 4159 tablet_copy_client.cc:647] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:25.205111 3059 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.46: config changed from index -1 to 46, NON_VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) added. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: true } } }
I20240501 07:59:25.207305 2536 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.46: config changed from index -1 to 46, NON_VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) added. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: true } } }
I20240501 07:59:25.208608 2259 catalog_manager.cc:5484] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 reported cstate change: config changed from index -1 to 46, NON_VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: true } } }
I20240501 07:59:25.211562 4159 tablet_copy_client.cc:523] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:25.221866 2407 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 45. Preceding OpId from leader: term: 1 index: 46. (index mismatch)
I20240501 07:59:25.227051 4159 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: Bootstrap starting.
I20240501 07:59:25.233439 4158 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 44, Time since last communication: 0.000s
I20240501 07:59:25.238363 2799 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.46: config changed from index -1 to 46, NON_VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) added. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: true } } }
I20240501 07:59:25.269124 2667 consensus_queue.cc:237] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 63, Committed index: 63, Last appended: 1.63, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:25.271137 2933 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
W20240501 07:59:25.271457 2606 consensus_peers.cc:485] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df -> Peer e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389): Couldn't send request to peer e48105f8f6ab476196dd66c4b12808d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ade42f3f20e4deda01e89ee6ee4141f. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:25.271855 3544 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20240501 07:59:25.273125 2537 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 07:59:25.273496 3544 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20240501 07:59:25.276865 4158 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 46, Last known committed idx: 44, Time since last communication: 0.000s
I20240501 07:59:25.277922 4084 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.64: config changed from index -1 to 64, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:25.281706 2539 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index -1 to 64, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:25.285311 2407 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 07:59:25.285701 4196 raft_consensus.cc:1050] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2: attempting to promote NON_VOTER e48105f8f6ab476196dd66c4b12808d8 to VOTER
I20240501 07:59:25.286137 4196 consensus_queue.cc:237] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 65, Committed index: 65, Last appended: 1.65, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:25.286787 4168 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=33 overwritten=0 applied=33 ignored=0} inserts{seen=718 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:25.287075 4168 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap complete.
I20240501 07:59:25.287391 4168 ts_tablet_manager.cc:1367] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8: Time spent bootstrapping tablet: real 0.216s user 0.044s sys 0.004s
I20240501 07:59:25.287976 4168 raft_consensus.cc:348] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:25.288108 4168 raft_consensus.cc:729] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e48105f8f6ab476196dd66c4b12808d8, State: Initialized, Role: LEARNER
I20240501 07:59:25.288211 4168 consensus_queue.cc:260] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 33, Last appended: 1.33, Last appended by leader: 33, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 28 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:25.288471 4168 ts_tablet_manager.cc:1398] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:25.291240 2259 catalog_manager.cc:5484] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index -1 to 64, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:25.291390 2429 tablet_copy_service.cc:342] P 85377a976c05408fbb384237a0edb691: Request end of tablet copy session e48105f8f6ab476196dd66c4b12808d8-1ae5c568fb854f2a9e4122fc21314024 received from {username='slave'} at 127.0.99.135:52723
I20240501 07:59:25.291471 2429 tablet_copy_service.cc:433] P 85377a976c05408fbb384237a0edb691: ending tablet copy session e48105f8f6ab476196dd66c4b12808d8-1ae5c568fb854f2a9e4122fc21314024 on tablet 1ae5c568fb854f2a9e4122fc21314024 with peer e48105f8f6ab476196dd66c4b12808d8
I20240501 07:59:25.295182 2930 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index -1 to 64, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:25.300779 2931 consensus_queue.cc:237] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 51, Committed index: 51, Last appended: 1.51, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:25.302134 3063 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 51. Preceding OpId from leader: term: 1 index: 52. (index mismatch)
I20240501 07:59:25.302498 4102 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 52, Last known committed idx: 51, Time since last communication: 0.000s
I20240501 07:59:25.306936 2667 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 51. Preceding OpId from leader: term: 1 index: 52. (index mismatch)
I20240501 07:59:25.308878 2537 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 51. Preceding OpId from leader: term: 1 index: 52. (index mismatch)
I20240501 07:59:25.312175 4102 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 52, Last known committed idx: 51, Time since last communication: 0.000s
I20240501 07:59:25.312428 4102 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 52, Last known committed idx: 50, Time since last communication: 0.000s
I20240501 07:59:25.316128 4102 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.52: config changed from index -1 to 52, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:25.319077 2535 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.52: config changed from index -1 to 52, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:25.321596 3059 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.52: config changed from index -1 to 52, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:25.322906 2667 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.52: config changed from index -1 to 52, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:25.323917 2259 catalog_manager.cc:5484] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c reported cstate change: config changed from index -1 to 52, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:25.328274 3059 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 07:59:25.330212 4174 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20240501 07:59:25.331317 3059 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index -1 to 64, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:25.333894 4174 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 63, Time since last communication: 0.000s
I20240501 07:59:25.337318 2407 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index -1 to 64, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:25.344521 2407 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.46: config changed from index -1 to 46, NON_VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) added. New config: { opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: true } } }
I20240501 07:59:25.348153 2798 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 51. Preceding OpId from leader: term: 1 index: 52. (index mismatch)
W20240501 07:59:25.349089 2868 consensus_peers.cc:485] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 -> Peer e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389): Couldn't send request to peer e48105f8f6ab476196dd66c4b12808d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a42ea1bbca294f3e9298d895eee087c7. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:25.350224 4194 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 52, Last known committed idx: 50, Time since last communication: 0.000s
I20240501 07:59:25.352679 2800 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.52: config changed from index -1 to 52, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:25.360924 2535 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240501 07:59:25.362470 3995 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240501 07:59:25.369676 3063 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240501 07:59:25.370296 4194 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240501 07:59:25.379967 3870 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240501 07:59:25.382437 2405 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240501 07:59:25.383565 4194 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240501 07:59:25.385417 2800 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 64. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240501 07:59:25.385532 4194 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 64, Time since last communication: 0.000s
I20240501 07:59:25.399991 4197 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.66: config changed from index 37 to 66, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.417181 2537 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 37 to 66, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.420451 2265 catalog_manager.cc:5484] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c reported cstate change: config changed from index 37 to 66, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.422806 3060 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 37 to 66, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.425419 4196 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 63, Time since last communication: 0.000s
I20240501 07:59:25.442656 3870 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 37 to 66, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.454192 3870 raft_consensus.cc:1203] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.27->[1.28-1.38] Dedup: 1.33->[1.34-1.38]
I20240501 07:59:25.457638 2802 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 37 to 66, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.485841 2930 consensus_queue.cc:237] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 67, Committed index: 67, Last appended: 1.68, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:25.487360 3059 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 68. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240501 07:59:25.487754 4196 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 67, Time since last communication: 0.000s
I20240501 07:59:25.499367 3869 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 68. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240501 07:59:25.499812 4196 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 67, Time since last communication: 0.000s
I20240501 07:59:25.503046 2799 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 68. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240501 07:59:25.503530 4003 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.69: config changed from index 66 to 69, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.506224 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f64014c3e7474e21b65082390fc1e6a6 with cas_config_opid_index 66: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:25.506304 2260 catalog_manager.cc:5484] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: config changed from index 66 to 69, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:25.507040 3060 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 66 to 69, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.509805 3869 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 66 to 69, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.510155 2520 tablet_service.cc:1514] Processing DeleteTablet for tablet f64014c3e7474e21b65082390fc1e6a6 with delete_type TABLET_DATA_TOMBSTONED (TS 5810fd56ce9244438edcd3c0ea23e72c not found in new config with opid_index 69) from {username='slave'} at 127.0.0.1:56996
I20240501 07:59:25.511003 3995 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 67, Time since last communication: 0.000s
I20240501 07:59:25.511812 4238 tablet_replica.cc:330] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c: stopping tablet replica
I20240501 07:59:25.512171 4238 raft_consensus.cc:2229] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:25.512399 4238 pending_rounds.cc:62] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c: Trying to abort 1 pending ops.
I20240501 07:59:25.512482 4238 pending_rounds.cc:68] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c: Aborting op as it isn't in flight: id { term: 1 index: 68 } timestamp: 7022798296844836864 op_type: WRITE_OP write_request { tablet_id: "f64014c3e7474e21b65082390fc1e6a6" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\0004\302=O\370\362\0079\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""502c2eb2095b60d3" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7022798296813776896 authz_token { token_data: "\010\206\357\307\261\006\"3\n\005slave\022*\n d87957cf347545a38a6298fa134ae83b\020\001\030\001 \001(\001" signature: "<redacted>""\2400s\310\022&\252rl\314\'\261\331q~\014n6\264:\177\246\252\016\356\017\245\265\212\251\023\376\232\322E\334(5\315\313\277\336\356\236\201\247\311\033G\371FN@\345T\\\254\377BA-\2575>" signing_key_seq_num: 0 } } request_id { client_id: "3028ad60c3044f3692b88afcc718ce9a" seq_no: 158 first_incomplete_seq_no: 158 attempt_no: 0 }
I20240501 07:59:25.512847 4238 raft_consensus.cc:2258] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:25.514190 4238 ts_tablet_manager.cc:1875] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:25.515054 2802 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 66 to 69, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.520407 4238 ts_tablet_manager.cc:1888] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.68
I20240501 07:59:25.520531 4238 log.cc:1196] T f64014c3e7474e21b65082390fc1e6a6 P 5810fd56ce9244438edcd3c0ea23e72c: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/f64014c3e7474e21b65082390fc1e6a6
I20240501 07:59:25.521674 2251 catalog_manager.cc:4830] TS 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773): tablet f64014c3e7474e21b65082390fc1e6a6 (table rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]) successfully deleted
I20240501 07:59:25.532457 4233 raft_consensus.cc:1050] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691: attempting to promote NON_VOTER e48105f8f6ab476196dd66c4b12808d8 to VOTER
I20240501 07:59:25.533003 4233 consensus_queue.cc:237] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 39, Committed index: 39, Last appended: 1.39, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:25.535532 2670 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 39. Preceding OpId from leader: term: 1 index: 40. (index mismatch)
I20240501 07:59:25.535681 2802 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 39. Preceding OpId from leader: term: 1 index: 40. (index mismatch)
I20240501 07:59:25.536010 4067 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20240501 07:59:25.536134 3961 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20240501 07:59:25.536219 3869 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 39. Preceding OpId from leader: term: 1 index: 40. (index mismatch)
I20240501 07:59:25.536926 4067 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20240501 07:59:25.539314 4067 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Committing config change with OpId 1.40: config changed from index 28 to 40, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.541349 2670 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.40: config changed from index 28 to 40, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.541920 2930 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 39. Preceding OpId from leader: term: 1 index: 40. (index mismatch)
I20240501 07:59:25.542368 2799 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.40: config changed from index 28 to 40, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.542493 2260 catalog_manager.cc:5484] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index 28 to 40, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:25.547191 3961 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 39, Time since last communication: 0.000s
I20240501 07:59:25.549397 3869 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.40: config changed from index 28 to 40, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.554168 2930 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.40: config changed from index 28 to 40, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.559715 2404 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 37 to 66, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.561780 3059 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 39. Preceding OpId from leader: term: 1 index: 40. (index mismatch)
I20240501 07:59:25.565369 3553 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 40, Last known committed idx: 38, Time since last communication: 0.000s
I20240501 07:59:25.565721 2407 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 68. Preceding OpId from leader: term: 1 index: 69. (index mismatch)
I20240501 07:59:25.566466 3059 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.40: config changed from index 28 to 40, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 40 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.576728 4159 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=50 overwritten=0 applied=50 ignored=0} inserts{seen=735 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:25.577036 4159 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: Bootstrap complete.
I20240501 07:59:25.577237 4159 ts_tablet_manager.cc:1367] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: Time spent bootstrapping tablet: real 0.350s user 0.058s sys 0.000s
I20240501 07:59:25.577838 4159 raft_consensus.cc:348] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:25.577977 4159 raft_consensus.cc:729] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e48105f8f6ab476196dd66c4b12808d8, State: Initialized, Role: LEARNER
I20240501 07:59:25.578083 4159 consensus_queue.cc:260] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 50, Last appended: 1.50, Last appended by leader: 50, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:25.584349 4159 ts_tablet_manager.cc:1398] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240501 07:59:25.586032 4158 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 69, Last known committed idx: 67, Time since last communication: 0.000s
I20240501 07:59:25.597553 2691 tablet_copy_service.cc:342] P 5dd2fd1701284a6b886ab8d14d1bd8df: Request end of tablet copy session e48105f8f6ab476196dd66c4b12808d8-5414588c25d048ed84ec9aae6f300ddf received from {username='slave'} at 127.0.99.135:36749
I20240501 07:59:25.597661 2691 tablet_copy_service.cc:433] P 5dd2fd1701284a6b886ab8d14d1bd8df: ending tablet copy session e48105f8f6ab476196dd66c4b12808d8-5414588c25d048ed84ec9aae6f300ddf on tablet 5414588c25d048ed84ec9aae6f300ddf with peer e48105f8f6ab476196dd66c4b12808d8
I20240501 07:59:25.624797 2404 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.69: config changed from index 66 to 69, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 69 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.634516 2403 consensus_queue.cc:237] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 41, Committed index: 41, Last appended: 1.41, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:25.634933 4239 ts_tablet_manager.cc:902] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c: Initiating tablet copy from peer 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519)
I20240501 07:59:25.635861 4239 tablet_copy_client.cc:308] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:44519
I20240501 07:59:25.636574 3058 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 41. Preceding OpId from leader: term: 1 index: 42. (index mismatch)
I20240501 07:59:25.637144 2670 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 41. Preceding OpId from leader: term: 1 index: 42. (index mismatch)
I20240501 07:59:25.637460 3869 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 41. Preceding OpId from leader: term: 1 index: 42. (index mismatch)
I20240501 07:59:25.638947 4233 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20240501 07:59:25.640614 3869 raft_consensus.cc:1203] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.44->[1.45-1.56] Dedup: 1.50->[1.51-1.56]
I20240501 07:59:25.641638 3553 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20240501 07:59:25.645318 4067 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 41, Time since last communication: 0.000s
I20240501 07:59:25.645205 3961 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Committing config change with OpId 1.42: config changed from index 40 to 42, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.645596 2670 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.42: config changed from index 40 to 42, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.646855 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1ae5c568fb854f2a9e4122fc21314024 with cas_config_opid_index 40: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:25.647444 3062 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.42: config changed from index 40 to 42, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.649960 2259 catalog_manager.cc:5484] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index 40 to 42, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:25.651818 3870 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.42: config changed from index 40 to 42, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.653751 2691 tablet_copy_service.cc:140] P 5dd2fd1701284a6b886ab8d14d1bd8df: Received BeginTabletCopySession request for tablet 490523aeb9e44f4ba8c2fc42fb73a95e from peer 5810fd56ce9244438edcd3c0ea23e72c ({username='slave'} at 127.0.99.130:33345)
I20240501 07:59:25.653846 2691 tablet_copy_service.cc:161] P 5dd2fd1701284a6b886ab8d14d1bd8df: Beginning new tablet copy session on tablet 490523aeb9e44f4ba8c2fc42fb73a95e from peer 5810fd56ce9244438edcd3c0ea23e72c at {username='slave'} at 127.0.99.130:33345: session id = 5810fd56ce9244438edcd3c0ea23e72c-490523aeb9e44f4ba8c2fc42fb73a95e
I20240501 07:59:25.654328 2782 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ae5c568fb854f2a9e4122fc21314024 with delete_type TABLET_DATA_TOMBSTONED (TS 3bd6898f790b48d593253180b79c5e40 not found in new config with opid_index 42) from {username='slave'} at 127.0.0.1:51830
I20240501 07:59:25.654742 2691 tablet_copy_source_session.cc:205] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:25.667169 4252 tablet_replica.cc:330] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40: stopping tablet replica
I20240501 07:59:25.667780 4252 raft_consensus.cc:2229] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:25.668174 4252 raft_consensus.cc:2258] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:25.669706 4252 ts_tablet_manager.cc:1875] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:25.676052 4252 ts_tablet_manager.cc:1888] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.41
I20240501 07:59:25.676146 4252 log.cc:1196] T 1ae5c568fb854f2a9e4122fc21314024 P 3bd6898f790b48d593253180b79c5e40: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/1ae5c568fb854f2a9e4122fc21314024
I20240501 07:59:25.677615 2253 catalog_manager.cc:4830] TS 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829): tablet 1ae5c568fb854f2a9e4122fc21314024 (table rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]) successfully deleted
I20240501 07:59:25.714206 4239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 490523aeb9e44f4ba8c2fc42fb73a95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:25.723583 4239 tablet_copy_client.cc:783] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:25.723800 4239 tablet_copy_client.cc:647] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:25.743880 4159 ts_tablet_manager.cc:902] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8: Initiating tablet copy from peer 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519)
I20240501 07:59:25.744369 4159 tablet_copy_client.cc:308] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:44519
I20240501 07:59:25.762218 2691 tablet_copy_service.cc:140] P 5dd2fd1701284a6b886ab8d14d1bd8df: Received BeginTabletCopySession request for tablet 3ade42f3f20e4deda01e89ee6ee4141f from peer e48105f8f6ab476196dd66c4b12808d8 ({username='slave'} at 127.0.99.135:36749)
I20240501 07:59:25.762331 2691 tablet_copy_service.cc:161] P 5dd2fd1701284a6b886ab8d14d1bd8df: Beginning new tablet copy session on tablet 3ade42f3f20e4deda01e89ee6ee4141f from peer e48105f8f6ab476196dd66c4b12808d8 at {username='slave'} at 127.0.99.135:36749: session id = e48105f8f6ab476196dd66c4b12808d8-3ade42f3f20e4deda01e89ee6ee4141f
I20240501 07:59:25.763234 2691 tablet_copy_source_session.cc:205] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:25.764231 4239 tablet_copy_client.cc:523] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:25.765977 4239 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap starting.
I20240501 07:59:25.784232 4159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ade42f3f20e4deda01e89ee6ee4141f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:25.786401 4159 tablet_copy_client.cc:783] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:25.786564 4159 tablet_copy_client.cc:647] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:25.790540 2931 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 40. Preceding OpId from leader: term: 1 index: 42. (index mismatch)
I20240501 07:59:25.799582 4148 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 42, Last known committed idx: 40, Time since last communication: 0.000s
I20240501 07:59:25.803509 4275 ts_tablet_manager.cc:902] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df: Initiating tablet copy from peer e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591)
I20240501 07:59:25.805356 4275 tablet_copy_client.cc:308] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:35591
I20240501 07:59:25.805889 2953 tablet_copy_service.cc:140] P e2c508eb3dac4193a1eeba799e5d54f2: Received BeginTabletCopySession request for tablet 6949f70a1ae44576b42d388231f0b50f from peer 5dd2fd1701284a6b886ab8d14d1bd8df ({username='slave'} at 127.0.99.131:45987)
I20240501 07:59:25.805987 2953 tablet_copy_service.cc:161] P e2c508eb3dac4193a1eeba799e5d54f2: Beginning new tablet copy session on tablet 6949f70a1ae44576b42d388231f0b50f from peer 5dd2fd1701284a6b886ab8d14d1bd8df at {username='slave'} at 127.0.99.131:45987: session id = 5dd2fd1701284a6b886ab8d14d1bd8df-6949f70a1ae44576b42d388231f0b50f
I20240501 07:59:25.806856 2953 tablet_copy_source_session.cc:205] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:25.809286 4159 tablet_copy_client.cc:523] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:25.810649 4159 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8: Bootstrap starting.
I20240501 07:59:25.819779 4275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6949f70a1ae44576b42d388231f0b50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:25.822010 4275 tablet_copy_client.cc:783] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:25.822187 4275 tablet_copy_client.cc:647] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:25.826211 4275 tablet_copy_client.cc:523] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:25.828187 4275 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap starting.
I20240501 07:59:25.828787 4020 raft_consensus.cc:1050] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df: attempting to promote NON_VOTER e48105f8f6ab476196dd66c4b12808d8 to VOTER
I20240501 07:59:25.829293 4020 consensus_queue.cc:237] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 59, Committed index: 59, Last appended: 1.60, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:25.830435 2931 raft_consensus.cc:1203] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.40->[1.41-1.42] Dedup: 1.41->[1.42-1.42]
I20240501 07:59:25.830912 2931 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.42: config changed from index 40 to 42, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 42 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.831650 2928 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 59. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 07:59:25.832681 2800 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 59. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 07:59:25.832684 4018 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 58, Time since last communication: 0.000s
I20240501 07:59:25.834228 2538 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 07:59:25.834671 4018 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 59, Time since last communication: 0.000s
I20240501 07:59:25.834960 4018 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 59, Time since last communication: 0.000s
I20240501 07:59:25.834987 2408 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 59. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 07:59:25.835664 3870 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 59. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 07:59:25.837527 4018 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 59, Time since last communication: 0.000s
I20240501 07:59:25.838166 4020 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 58, Time since last communication: 0.000s
I20240501 07:59:25.843147 4018 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.61: config changed from index 45 to 61, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.845048 3870 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index 45 to 61, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.847779 2538 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index 45 to 61, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.850178 2265 catalog_manager.cc:5484] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 reported cstate change: config changed from index 45 to 61, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.850715 2404 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index 45 to 61, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.850823 2798 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index 45 to 61, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.851140 2930 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index 45 to 61, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.863188 2669 consensus_queue.cc:237] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 61, Committed index: 61, Last appended: 1.61, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:25.864573 2535 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 62. (index mismatch)
I20240501 07:59:25.864832 2928 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 62. (index mismatch)
I20240501 07:59:25.864918 4174 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 61, Time since last communication: 0.000s
I20240501 07:59:25.865540 2799 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 62. (index mismatch)
I20240501 07:59:25.865820 4174 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 61, Time since last communication: 0.000s
I20240501 07:59:25.868216 3870 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 62. (index mismatch)
I20240501 07:59:25.869202 4020 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 61, Time since last communication: 0.000s
I20240501 07:59:25.869431 3740 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 61, Time since last communication: 0.000s
I20240501 07:59:25.870019 4174 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.62: config changed from index 61 to 62, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.870324 2928 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.62: config changed from index 61 to 62, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.872121 2252 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5414588c25d048ed84ec9aae6f300ddf with cas_config_opid_index 61: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:25.872123 2539 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.62: config changed from index 61 to 62, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.872884 2260 catalog_manager.cc:5484] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index 61 to 62, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:25.880105 2868 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.135:37389, user_credentials={real_user=slave}} blocked reactor thread for 56803.1us
I20240501 07:59:25.884194 2798 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.62: config changed from index 61 to 62, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.889212 3869 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.62: config changed from index 61 to 62, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:25.898141 2387 tablet_service.cc:1514] Processing DeleteTablet for tablet 5414588c25d048ed84ec9aae6f300ddf with delete_type TABLET_DATA_TOMBSTONED (TS 85377a976c05408fbb384237a0edb691 not found in new config with opid_index 62) from {username='slave'} at 127.0.0.1:48528
I20240501 07:59:25.914882 4290 tablet_replica.cc:330] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691: stopping tablet replica
I20240501 07:59:25.915303 4290 raft_consensus.cc:2229] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:25.919216 4290 raft_consensus.cc:2258] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:25.927668 4239 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap replayed 1/1 log segments. Stats: ops{read=50 overwritten=0 applied=50 ignored=0} inserts{seen=722 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:25.928050 4239 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c: Bootstrap complete.
I20240501 07:59:25.928282 4239 ts_tablet_manager.cc:1367] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c: Time spent bootstrapping tablet: real 0.162s user 0.056s sys 0.005s
I20240501 07:59:25.928735 4239 raft_consensus.cc:348] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: true } }
I20240501 07:59:25.928948 4239 raft_consensus.cc:729] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5810fd56ce9244438edcd3c0ea23e72c, State: Initialized, Role: LEARNER
I20240501 07:59:25.929112 4239 consensus_queue.cc:260] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 50, Last appended: 1.50, Last appended by leader: 50, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: true } }
I20240501 07:59:25.929510 4239 ts_tablet_manager.cc:1398] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:25.939730 2691 tablet_copy_service.cc:342] P 5dd2fd1701284a6b886ab8d14d1bd8df: Request end of tablet copy session 5810fd56ce9244438edcd3c0ea23e72c-490523aeb9e44f4ba8c2fc42fb73a95e received from {username='slave'} at 127.0.99.130:33345
I20240501 07:59:25.939818 2691 tablet_copy_service.cc:433] P 5dd2fd1701284a6b886ab8d14d1bd8df: ending tablet copy session 5810fd56ce9244438edcd3c0ea23e72c-490523aeb9e44f4ba8c2fc42fb73a95e on tablet 490523aeb9e44f4ba8c2fc42fb73a95e with peer 5810fd56ce9244438edcd3c0ea23e72c
I20240501 07:59:25.941094 4290 ts_tablet_manager.cc:1875] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:25.943074 4281 ts_tablet_manager.cc:902] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8: Initiating tablet copy from peer e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591)
I20240501 07:59:25.945302 4281 tablet_copy_client.cc:308] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:35591
I20240501 07:59:25.972879 4290 ts_tablet_manager.cc:1888] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.61
I20240501 07:59:25.973004 4290 log.cc:1196] T 5414588c25d048ed84ec9aae6f300ddf P 85377a976c05408fbb384237a0edb691: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/5414588c25d048ed84ec9aae6f300ddf
I20240501 07:59:25.974218 2253 catalog_manager.cc:4830] TS 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933): tablet 5414588c25d048ed84ec9aae6f300ddf (table rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]) successfully deleted
I20240501 07:59:26.025151 2953 tablet_copy_service.cc:140] P e2c508eb3dac4193a1eeba799e5d54f2: Received BeginTabletCopySession request for tablet a42ea1bbca294f3e9298d895eee087c7 from peer e48105f8f6ab476196dd66c4b12808d8 ({username='slave'} at 127.0.99.135:50307)
I20240501 07:59:26.025269 2953 tablet_copy_service.cc:161] P e2c508eb3dac4193a1eeba799e5d54f2: Beginning new tablet copy session on tablet a42ea1bbca294f3e9298d895eee087c7 from peer e48105f8f6ab476196dd66c4b12808d8 at {username='slave'} at 127.0.99.135:50307: session id = e48105f8f6ab476196dd66c4b12808d8-a42ea1bbca294f3e9298d895eee087c7
I20240501 07:59:26.026199 2953 tablet_copy_source_session.cc:205] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:26.027130 4281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42ea1bbca294f3e9298d895eee087c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:26.029865 4281 tablet_copy_client.cc:783] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:26.030032 4281 tablet_copy_client.cc:647] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:26.056506 4281 tablet_copy_client.cc:523] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:26.057904 4281 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap starting.
I20240501 07:59:26.085028 4275 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap replayed 1/1 log segments. Stats: ops{read=54 overwritten=0 applied=54 ignored=0} inserts{seen=739 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:26.085422 4275 tablet_bootstrap.cc:492] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df: Bootstrap complete.
I20240501 07:59:26.085664 4275 ts_tablet_manager.cc:1367] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent bootstrapping tablet: real 0.258s user 0.048s sys 0.013s
I20240501 07:59:26.088508 4159 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=69 overwritten=0 applied=69 ignored=0} inserts{seen=741 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:26.088819 4159 tablet_bootstrap.cc:492] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8: Bootstrap complete.
I20240501 07:59:26.089040 4159 ts_tablet_manager.cc:1367] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8: Time spent bootstrapping tablet: real 0.278s user 0.072s sys 0.000s
I20240501 07:59:26.089509 4159 raft_consensus.cc:348] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:26.089699 4159 raft_consensus.cc:729] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e48105f8f6ab476196dd66c4b12808d8, State: Initialized, Role: LEARNER
I20240501 07:59:26.089828 4159 consensus_queue.cc:260] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 69, Last appended: 1.69, Last appended by leader: 69, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:26.092630 4159 ts_tablet_manager.cc:1398] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
W20240501 07:59:26.094914 2607 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.134:44027, user_credentials={real_user=slave}} blocked reactor thread for 44150.3us
I20240501 07:59:26.098996 4275 raft_consensus.cc:348] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: true } }
I20240501 07:59:26.099121 4275 raft_consensus.cc:729] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5dd2fd1701284a6b886ab8d14d1bd8df, State: Initialized, Role: LEARNER
I20240501 07:59:26.099215 4275 consensus_queue.cc:260] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 54, Last appended: 1.54, Last appended by leader: 54, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 46 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: true } }
I20240501 07:59:26.099682 4275 ts_tablet_manager.cc:1398] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df: Time spent starting tablet: real 0.014s user 0.001s sys 0.000s
I20240501 07:59:26.103782 2691 tablet_copy_service.cc:342] P 5dd2fd1701284a6b886ab8d14d1bd8df: Request end of tablet copy session e48105f8f6ab476196dd66c4b12808d8-3ade42f3f20e4deda01e89ee6ee4141f received from {username='slave'} at 127.0.99.135:36749
I20240501 07:59:26.103859 2691 tablet_copy_service.cc:433] P 5dd2fd1701284a6b886ab8d14d1bd8df: ending tablet copy session e48105f8f6ab476196dd66c4b12808d8-3ade42f3f20e4deda01e89ee6ee4141f on tablet 3ade42f3f20e4deda01e89ee6ee4141f with peer e48105f8f6ab476196dd66c4b12808d8
I20240501 07:59:26.104672 2953 tablet_copy_service.cc:342] P e2c508eb3dac4193a1eeba799e5d54f2: Request end of tablet copy session 5dd2fd1701284a6b886ab8d14d1bd8df-6949f70a1ae44576b42d388231f0b50f received from {username='slave'} at 127.0.99.131:45987
I20240501 07:59:26.104745 2953 tablet_copy_service.cc:433] P e2c508eb3dac4193a1eeba799e5d54f2: ending tablet copy session 5dd2fd1701284a6b886ab8d14d1bd8df-6949f70a1ae44576b42d388231f0b50f on tablet 6949f70a1ae44576b42d388231f0b50f with peer 5dd2fd1701284a6b886ab8d14d1bd8df
I20240501 07:59:26.105568 2931 consensus_queue.cc:237] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 52, Committed index: 52, Last appended: 1.52, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:26.107226 2538 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240501 07:59:26.107584 4197 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 52, Time since last communication: 0.000s
I20240501 07:59:26.114260 2538 raft_consensus.cc:1203] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.43->[1.44-1.55] Dedup: 1.50->[1.51-1.55]
I20240501 07:59:26.118407 3063 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240501 07:59:26.120342 2669 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240501 07:59:26.120780 4158 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 52, Time since last communication: 0.000s
I20240501 07:59:26.126744 4158 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 52, Time since last communication: 0.000s
I20240501 07:59:26.130089 3995 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:26.132172 2539 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:26.134704 2669 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:26.134840 2260 catalog_manager.cc:5484] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c reported cstate change: config changed from index -1 to 53, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:26.169194 3062 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:26.201133 3869 raft_consensus.cc:1203] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.63->[1.64-1.70] Dedup: 1.69->[1.70-1.70]
I20240501 07:59:26.232210 2799 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240501 07:59:26.247743 3989 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 52, Time since last communication: 0.000s
I20240501 07:59:26.274236 4020 raft_consensus.cc:1050] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df: attempting to promote NON_VOTER e48105f8f6ab476196dd66c4b12808d8 to VOTER
I20240501 07:59:26.274870 4020 consensus_queue.cc:237] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 71, Committed index: 71, Last appended: 1.71, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:26.275323 2799 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:26.276525 3869 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 71. Preceding OpId from leader: term: 1 index: 72. (index mismatch)
I20240501 07:59:26.278384 3062 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 71. Preceding OpId from leader: term: 1 index: 72. (index mismatch)
I20240501 07:59:26.279330 2536 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 71. Preceding OpId from leader: term: 1 index: 72. (index mismatch)
I20240501 07:59:26.279330 3740 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 71, Time since last communication: 0.000s
I20240501 07:59:26.280149 2929 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 71. Preceding OpId from leader: term: 1 index: 72. (index mismatch)
I20240501 07:59:26.280647 3740 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 71, Time since last communication: 0.000s
I20240501 07:59:26.280989 4298 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 71, Time since last communication: 0.000s
I20240501 07:59:26.281275 3616 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 71, Time since last communication: 0.000s
I20240501 07:59:26.285804 4018 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.72: config changed from index 64 to 72, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:26.286252 3059 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.72: config changed from index 64 to 72, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
W20240501 07:59:26.286927 2868 consensus_peers.cc:485] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 -> Peer e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389): Couldn't send request to peer e48105f8f6ab476196dd66c4b12808d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 917c88788fbf4089a5df0f271cc478f7. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:26.290535 2259 catalog_manager.cc:5484] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 reported cstate change: config changed from index 64 to 72, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:26.293678 2403 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 71. Preceding OpId from leader: term: 1 index: 72. (index mismatch)
I20240501 07:59:26.310231 2535 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.72: config changed from index 64 to 72, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:26.312711 3869 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.72: config changed from index 64 to 72, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:26.317934 4174 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 71, Time since last communication: 0.000s
I20240501 07:59:26.325402 2404 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.72: config changed from index 64 to 72, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:26.351023 2670 raft_consensus.cc:1203] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.45->[1.46-1.55] Dedup: 1.54->[1.55-1.55]
I20240501 07:59:26.360474 4174 raft_consensus.cc:1050] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df: attempting to promote NON_VOTER 5810fd56ce9244438edcd3c0ea23e72c to VOTER
I20240501 07:59:26.360853 4174 consensus_queue.cc:237] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 61, Committed index: 61, Last appended: 1.62, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } }
I20240501 07:59:26.366384 3059 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 62. Preceding OpId from leader: term: 1 index: 63. (index mismatch)
I20240501 07:59:26.370958 3544 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 63, Last known committed idx: 61, Time since last communication: 0.000s
I20240501 07:59:26.384303 2535 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 LEARNER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 62. Preceding OpId from leader: term: 1 index: 63. (index mismatch)
I20240501 07:59:26.384874 3544 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 63, Last known committed idx: 61, Time since last communication: 0.000s
I20240501 07:59:26.397732 2799 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 63. (index mismatch)
I20240501 07:59:26.398752 4173 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 63, Last known committed idx: 61, Time since last communication: 0.000s
I20240501 07:59:26.399281 2928 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 63. (index mismatch)
I20240501 07:59:26.400380 2408 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 62. Preceding OpId from leader: term: 1 index: 63. (index mismatch)
I20240501 07:59:26.400378 4173 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 63, Last known committed idx: 61, Time since last communication: 0.000s
I20240501 07:59:26.411330 3544 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.63: config changed from index 44 to 63, 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } }
I20240501 07:59:26.415877 2539 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.63: config changed from index 44 to 63, 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } }
I20240501 07:59:26.420589 4174 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 63, Last known committed idx: 61, Time since last communication: 0.000s
I20240501 07:59:26.421748 2259 catalog_manager.cc:5484] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index 44 to 63, 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:26.426106 2798 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.63: config changed from index 44 to 63, 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } }
I20240501 07:59:26.438275 2929 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.63: config changed from index 44 to 63, 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } }
I20240501 07:59:26.442281 4158 raft_consensus.cc:1050] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2: attempting to promote NON_VOTER 5dd2fd1701284a6b886ab8d14d1bd8df to VOTER
I20240501 07:59:26.442596 4158 consensus_queue.cc:237] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 57, Committed index: 57, Last appended: 1.58, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } }
I20240501 07:59:26.445792 2538 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 07:59:26.446558 3989 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 57, Time since last communication: 0.000s
I20240501 07:59:26.447391 2802 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 07:59:26.447734 3989 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 57, Time since last communication: 0.000s
I20240501 07:59:26.458529 2403 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.63: config changed from index 44 to 63, 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } }
I20240501 07:59:26.468474 2929 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.72: config changed from index 64 to 72, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 72 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:26.471721 2667 consensus_queue.cc:237] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 72, Committed index: 72, Last appended: 1.73, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:26.473963 2928 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 07:59:26.474434 4020 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 07:59:26.475970 3868 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 07:59:26.477128 4174 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 07:59:26.477710 2408 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 07:59:26.481978 4020 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.74: config changed from index 72 to 74, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:26.483732 3062 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 07:59:26.484172 2252 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3ade42f3f20e4deda01e89ee6ee4141f with cas_config_opid_index 72: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:26.484508 2928 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 72 to 74, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:26.484632 3868 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 72 to 74, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:26.485090 3063 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.63: config changed from index 44 to 63, 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } }
I20240501 07:59:26.486990 2260 catalog_manager.cc:5484] T 3ade42f3f20e4deda01e89ee6ee4141f P e48105f8f6ab476196dd66c4b12808d8 reported cstate change: config changed from index 72 to 74, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:26.489640 3989 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 57, Time since last communication: 0.000s
I20240501 07:59:26.489969 3062 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 07:59:26.491386 4195 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.59: config changed from index 46 to 59, 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } } }
I20240501 07:59:26.491628 2670 consensus_queue.cc:237] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 63, Committed index: 63, Last appended: 1.64, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } }
I20240501 07:59:26.492529 4020 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 07:59:26.492470 3062 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 46 to 59, 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } } }
I20240501 07:59:26.493075 2538 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 46 to 59, 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } } }
I20240501 07:59:26.494370 2387 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ade42f3f20e4deda01e89ee6ee4141f with delete_type TABLET_DATA_TOMBSTONED (TS 85377a976c05408fbb384237a0edb691 not found in new config with opid_index 74) from {username='slave'} at 127.0.0.1:48528
I20240501 07:59:26.495833 2799 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 64. Preceding OpId from leader: term: 1 index: 65. (index mismatch)
I20240501 07:59:26.496733 4174 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 65, Last known committed idx: 63, Time since last communication: 0.000s
I20240501 07:59:26.496871 2928 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 64. Preceding OpId from leader: term: 1 index: 65. (index mismatch)
I20240501 07:59:26.497974 4174 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 65, Last known committed idx: 63, Time since last communication: 0.000s
I20240501 07:59:26.498277 2259 catalog_manager.cc:5484] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 reported cstate change: config changed from index 46 to 59, 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } } }
I20240501 07:59:26.503671 2669 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEARNER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 07:59:26.504421 4195 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 57, Time since last communication: 0.000s
I20240501 07:59:26.504662 3616 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.65: config changed from index 63 to 65, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } }
I20240501 07:59:26.506304 4158 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 57, Time since last communication: 0.000s
I20240501 07:59:26.506858 2252 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 490523aeb9e44f4ba8c2fc42fb73a95e with cas_config_opid_index 63: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:26.507164 2799 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 46 to 59, 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } } }
I20240501 07:59:26.509178 2264 catalog_manager.cc:5484] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index 63 to 65, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:26.509809 2670 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 46 to 59, 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } } }
I20240501 07:59:26.514673 2928 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.65: config changed from index 63 to 65, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } }
I20240501 07:59:26.515471 2798 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.65: config changed from index 63 to 65, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } }
I20240501 07:59:26.516255 3043 tablet_service.cc:1514] Processing DeleteTablet for tablet 490523aeb9e44f4ba8c2fc42fb73a95e with delete_type TABLET_DATA_TOMBSTONED (TS 1537338fbeea47ee81f7a210cd6df2f0 not found in new config with opid_index 65) from {username='slave'} at 127.0.0.1:60186
I20240501 07:59:26.516803 4313 tablet_replica.cc:330] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0: stopping tablet replica
I20240501 07:59:26.517093 4313 raft_consensus.cc:2229] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:26.517331 4313 raft_consensus.cc:2258] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:26.523283 2404 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 64. Preceding OpId from leader: term: 1 index: 65. (index mismatch)
I20240501 07:59:26.524046 2403 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 46 to 59, 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } } }
I20240501 07:59:26.525811 2535 raft_consensus.cc:1261] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 72. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 07:59:26.527417 2535 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 64. Preceding OpId from leader: term: 1 index: 65. (index mismatch)
I20240501 07:59:26.528172 3740 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 65, Last known committed idx: 63, Time since last communication: 0.000s
I20240501 07:59:26.528856 3740 consensus_queue.cc:1035] T 3ade42f3f20e4deda01e89ee6ee4141f P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 07:59:26.529320 3062 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 72 to 74, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:26.529754 3740 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 65, Last known committed idx: 63, Time since last communication: 0.000s
I20240501 07:59:26.533265 4281 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=59 overwritten=0 applied=59 ignored=0} inserts{seen=690 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:26.535295 4281 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap complete.
I20240501 07:59:26.535545 4281 ts_tablet_manager.cc:1367] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8: Time spent bootstrapping tablet: real 0.478s user 0.061s sys 0.003s
I20240501 07:59:26.536036 4281 raft_consensus.cc:348] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:26.536163 4281 raft_consensus.cc:729] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e48105f8f6ab476196dd66c4b12808d8, State: Initialized, Role: LEARNER
I20240501 07:59:26.536267 4281 consensus_queue.cc:260] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 59, Last appended: 1.59, Last appended by leader: 59, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 52 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:26.538259 4281 ts_tablet_manager.cc:1398] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:26.551190 2953 tablet_copy_service.cc:342] P e2c508eb3dac4193a1eeba799e5d54f2: Request end of tablet copy session e48105f8f6ab476196dd66c4b12808d8-a42ea1bbca294f3e9298d895eee087c7 received from {username='slave'} at 127.0.99.135:50307
I20240501 07:59:26.551276 2953 tablet_copy_service.cc:433] P e2c508eb3dac4193a1eeba799e5d54f2: ending tablet copy session e48105f8f6ab476196dd66c4b12808d8-a42ea1bbca294f3e9298d895eee087c7 on tablet a42ea1bbca294f3e9298d895eee087c7 with peer e48105f8f6ab476196dd66c4b12808d8
I20240501 07:59:26.551553 4159 ts_tablet_manager.cc:902] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8: Initiating tablet copy from peer e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591)
I20240501 07:59:26.552033 4159 tablet_copy_client.cc:308] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:35591
I20240501 07:59:26.554445 2539 raft_consensus.cc:1203] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.72->[1.73-1.74] Dedup: 1.73->[1.74-1.74]
I20240501 07:59:26.554774 2539 raft_consensus.cc:2932] T 3ade42f3f20e4deda01e89ee6ee4141f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 72 to 74, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:26.555784 4312 tablet_replica.cc:330] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691: stopping tablet replica
I20240501 07:59:26.556171 4312 raft_consensus.cc:2229] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:26.556478 4312 pending_rounds.cc:62] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691: Trying to abort 1 pending ops.
I20240501 07:59:26.556425 2538 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.65: config changed from index 63 to 65, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } }
I20240501 07:59:26.557014 4312 pending_rounds.cc:68] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691: Aborting op as it isn't in flight: id { term: 1 index: 73 } timestamp: 7022798301037502464 op_type: WRITE_OP write_request { tablet_id: "3ade42f3f20e4deda01e89ee6ee4141f" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\357\364\205\'\305\237\226C\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""54487eb046eabe6e" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7022798300891389952 authz_token { token_data: "\010\206\357\307\261\006\"3\n\005slave\022*\n de8b9d2a0ca34b6cbbbbd54c29e15b74\020\001\030\001 \001(\001" signature: "<redacted>""X$\n>$\217,\2065`\357#\035g\035\354\314\335\313\251\371\330\334w4\336l\370\003\023\215\331G\034\334\023X\365\006\376N^.\'\360\322\277\316\350{\334\322oz\023[\332{\t\361\302\023C\034" signing_key_seq_num: 0 } } request_id { client_id: "cd0459bade874dbaaeba09d0439fc8d3" seq_no: 194 first_incomplete_seq_no: 194 attempt_no: 0 }
I20240501 07:59:26.557683 4312 raft_consensus.cc:2258] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:26.557320 2403 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.65: config changed from index 63 to 65, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } }
I20240501 07:59:26.563527 4313 ts_tablet_manager.cc:1875] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:26.567291 4312 ts_tablet_manager.cc:1875] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:26.580066 4312 ts_tablet_manager.cc:1888] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.73
I20240501 07:59:26.580142 4312 log.cc:1196] T 3ade42f3f20e4deda01e89ee6ee4141f P 85377a976c05408fbb384237a0edb691: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/3ade42f3f20e4deda01e89ee6ee4141f
I20240501 07:59:26.592156 4313 ts_tablet_manager.cc:1888] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.63
I20240501 07:59:26.592267 4313 log.cc:1196] T 490523aeb9e44f4ba8c2fc42fb73a95e P 1537338fbeea47ee81f7a210cd6df2f0: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/490523aeb9e44f4ba8c2fc42fb73a95e
I20240501 07:59:26.594094 2253 catalog_manager.cc:4830] TS 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933): tablet 3ade42f3f20e4deda01e89ee6ee4141f (table rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]) successfully deleted
W20240501 07:59:26.606030 2868 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.129:42933, user_credentials={real_user=slave}} blocked reactor thread for 53175us
I20240501 07:59:26.609851 2254 catalog_manager.cc:4830] TS 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027): tablet 490523aeb9e44f4ba8c2fc42fb73a95e (table rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]) successfully deleted
I20240501 07:59:26.617296 2953 tablet_copy_service.cc:140] P e2c508eb3dac4193a1eeba799e5d54f2: Received BeginTabletCopySession request for tablet 917c88788fbf4089a5df0f271cc478f7 from peer e48105f8f6ab476196dd66c4b12808d8 ({username='slave'} at 127.0.99.135:50307)
I20240501 07:59:26.617404 2953 tablet_copy_service.cc:161] P e2c508eb3dac4193a1eeba799e5d54f2: Beginning new tablet copy session on tablet 917c88788fbf4089a5df0f271cc478f7 from peer e48105f8f6ab476196dd66c4b12808d8 at {username='slave'} at 127.0.99.135:50307: session id = e48105f8f6ab476196dd66c4b12808d8-917c88788fbf4089a5df0f271cc478f7
I20240501 07:59:26.618305 2953 tablet_copy_source_session.cc:205] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:26.624238 4159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917c88788fbf4089a5df0f271cc478f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:26.626358 4159 tablet_copy_client.cc:783] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:26.626510 4159 tablet_copy_client.cc:647] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:26.668330 4159 tablet_copy_client.cc:523] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:26.680241 4159 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap starting.
I20240501 07:59:26.865842 3868 raft_consensus.cc:1203] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.51->[1.52-1.72] Dedup: 1.59->[1.60-1.72]
I20240501 07:59:26.916859 2929 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f"
dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56766
I20240501 07:59:26.917024 2929 raft_consensus.cc:595] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Received request to transfer leadership
W20240501 07:59:26.920470 2893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:48556: Service unavailable: leader transfer in progress
W20240501 07:59:26.925927 2893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:48556: Service unavailable: leader transfer in progress
I20240501 07:59:26.926524 4197 raft_consensus.cc:979] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2: : Instructing follower 3bd6898f790b48d593253180b79c5e40 to start an election
I20240501 07:59:26.926622 4197 raft_consensus.cc:1067] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Signalling peer 3bd6898f790b48d593253180b79c5e40 to start an election
W20240501 07:59:26.929755 2893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:48556: Service unavailable: leader transfer in progress
I20240501 07:59:26.932283 2801 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f"
dest_uuid: "3bd6898f790b48d593253180b79c5e40"
from {username='slave'} at 127.0.99.133:58231
I20240501 07:59:26.932432 2801 raft_consensus.cc:482] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 07:59:26.932493 2801 raft_consensus.cc:3037] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Advancing to term 2
I20240501 07:59:26.933583 2801 raft_consensus.cc:504] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } }
I20240501 07:59:26.935041 2801 leader_election.cc:288] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [CANDIDATE]: Term 2 election: Requested vote from peers e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591), 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773), 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027), 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933), 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519)
W20240501 07:59:26.938236 2893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:48556: Service unavailable: leader transfer in progress
W20240501 07:59:26.948611 2893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:48556: Service unavailable: leader transfer in progress
W20240501 07:59:26.957135 2893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:48556: Service unavailable: leader transfer in progress
I20240501 07:59:26.968516 2930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "3bd6898f790b48d593253180b79c5e40" candidate_term: 2 candidate_status { last_received { term: 1 index: 66 } } ignore_live_leader: true dest_uuid: "e2c508eb3dac4193a1eeba799e5d54f2"
I20240501 07:59:26.968634 2930 raft_consensus.cc:3032] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 07:59:26.968673 2930 raft_consensus.cc:729] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e2c508eb3dac4193a1eeba799e5d54f2, State: Running, Role: LEADER
I20240501 07:59:26.968812 2930 consensus_queue.cc:260] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 66, Committed index: 66, Last appended: 1.66, Last appended by leader: 66, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } }
I20240501 07:59:26.969025 2930 raft_consensus.cc:3037] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Advancing to term 2
I20240501 07:59:26.970134 2930 raft_consensus.cc:2454] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd6898f790b48d593253180b79c5e40 in term 2.
W20240501 07:59:26.970968 2893 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:48556: Illegal state: replica e2c508eb3dac4193a1eeba799e5d54f2 is not leader of this config: current role FOLLOWER
I20240501 07:59:26.978391 2404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "3bd6898f790b48d593253180b79c5e40" candidate_term: 2 candidate_status { last_received { term: 1 index: 66 } } ignore_live_leader: true dest_uuid: "85377a976c05408fbb384237a0edb691"
I20240501 07:59:26.978521 2404 raft_consensus.cc:3037] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Advancing to term 2
I20240501 07:59:26.979607 2404 raft_consensus.cc:2454] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd6898f790b48d593253180b79c5e40 in term 2.
I20240501 07:59:26.982087 4196 raft_consensus.cc:1050] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2: attempting to promote NON_VOTER e48105f8f6ab476196dd66c4b12808d8 to VOTER
I20240501 07:59:26.984050 4196 consensus_queue.cc:237] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 74, Committed index: 74, Last appended: 1.74, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:26.986864 3062 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 74. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240501 07:59:26.987785 4158 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 74, Time since last communication: 0.000s
I20240501 07:59:26.987818 2538 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 74. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240501 07:59:26.988178 4158 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 74, Time since last communication: 0.000s
I20240501 07:59:26.995272 3868 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 74. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240501 07:59:26.995746 4159 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=54 overwritten=0 applied=54 ignored=0} inserts{seen=685 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:26.996434 2798 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 74. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240501 07:59:26.999146 4197 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 74, Time since last communication: 0.000s
I20240501 07:59:26.999779 4159 tablet_bootstrap.cc:492] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap complete.
I20240501 07:59:27.000033 4159 ts_tablet_manager.cc:1367] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8: Time spent bootstrapping tablet: real 0.320s user 0.053s sys 0.005s
I20240501 07:59:27.003064 4159 raft_consensus.cc:348] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:27.004011 2667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "3bd6898f790b48d593253180b79c5e40" candidate_term: 2 candidate_status { last_received { term: 1 index: 66 } } ignore_live_leader: true dest_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df"
I20240501 07:59:27.004119 2667 raft_consensus.cc:3037] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Advancing to term 2
I20240501 07:59:27.005164 2667 raft_consensus.cc:2454] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd6898f790b48d593253180b79c5e40 in term 2.
I20240501 07:59:27.003654 4159 raft_consensus.cc:729] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e48105f8f6ab476196dd66c4b12808d8, State: Initialized, Role: LEARNER
I20240501 07:59:27.005656 4159 consensus_queue.cc:260] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 54, Last appended: 1.54, Last appended by leader: 54, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:27.005967 4159 ts_tablet_manager.cc:1398] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20240501 07:59:27.008227 2667 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 74. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240501 07:59:27.010223 2736 leader_election.cc:302] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 6 voters: 4 yes votes; 0 no votes. yes voters: 3bd6898f790b48d593253180b79c5e40, 5dd2fd1701284a6b886ab8d14d1bd8df, 85377a976c05408fbb384237a0edb691, e2c508eb3dac4193a1eeba799e5d54f2; no voters:
I20240501 07:59:27.010772 4133 raft_consensus.cc:2781] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 07:59:27.010968 4133 raft_consensus.cc:686] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 2 LEADER]: Becoming Leader. State: Replica: 3bd6898f790b48d593253180b79c5e40, State: Running, Role: LEADER
I20240501 07:59:27.011260 4133 consensus_queue.cc:237] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 66, Committed index: 66, Last appended: 1.66, Last appended by leader: 66, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } }
I20240501 07:59:27.017261 2260 catalog_manager.cc:5484] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 reported cstate change: term changed from 1 to 2, leader changed from e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) to 3bd6898f790b48d593253180b79c5e40 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "3bd6898f790b48d593253180b79c5e40" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:27.019176 2953 tablet_copy_service.cc:342] P e2c508eb3dac4193a1eeba799e5d54f2: Request end of tablet copy session e48105f8f6ab476196dd66c4b12808d8-917c88788fbf4089a5df0f271cc478f7 received from {username='slave'} at 127.0.99.135:50307
I20240501 07:59:27.019259 2953 tablet_copy_service.cc:433] P e2c508eb3dac4193a1eeba799e5d54f2: ending tablet copy session e48105f8f6ab476196dd66c4b12808d8-917c88788fbf4089a5df0f271cc478f7 on tablet 917c88788fbf4089a5df0f271cc478f7 with peer e48105f8f6ab476196dd66c4b12808d8
I20240501 07:59:27.021901 4197 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 74, Time since last communication: 0.000s
I20240501 07:59:27.032512 4195 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.75: config changed from index 52 to 75, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.035590 2536 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 52 to 75, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.036391 3062 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 52 to 75, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.040971 2259 catalog_manager.cc:5484] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 reported cstate change: config changed from index 52 to 75, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.041805 3868 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 52 to 75, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.042956 3989 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 74, Time since last communication: 0.000s
I20240501 07:59:27.048785 2800 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 52 to 75, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.050566 2403 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 2 FOLLOWER]: Refusing update from remote peer 3bd6898f790b48d593253180b79c5e40: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 2 index: 68. (index mismatch)
I20240501 07:59:27.051046 2667 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 2 FOLLOWER]: Refusing update from remote peer 3bd6898f790b48d593253180b79c5e40: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 2 index: 68. (index mismatch)
I20240501 07:59:27.051813 4383 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240501 07:59:27.052212 4383 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240501 07:59:27.052412 2930 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 2 FOLLOWER]: Refusing update from remote peer 3bd6898f790b48d593253180b79c5e40: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 2 index: 68. (index mismatch)
I20240501 07:59:27.053505 4133 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240501 07:59:27.064165 2670 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 52 to 75, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.083906 3870 raft_consensus.cc:1203] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.52->[1.53-1.61] Dedup: 1.54->[1.55-1.61]
I20240501 07:59:27.109146 3059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "3bd6898f790b48d593253180b79c5e40" candidate_term: 2 candidate_status { last_received { term: 1 index: 66 } } ignore_live_leader: true dest_uuid: "1537338fbeea47ee81f7a210cd6df2f0"
I20240501 07:59:27.109279 3059 raft_consensus.cc:3037] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 07:59:27.110291 3059 raft_consensus.cc:2454] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd6898f790b48d593253180b79c5e40 in term 2.
I20240501 07:59:27.111161 3062 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 2 FOLLOWER]: Refusing update from remote peer 3bd6898f790b48d593253180b79c5e40: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 2 index: 68. (index mismatch)
I20240501 07:59:27.111806 4387 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240501 07:59:27.130897 2539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6949f70a1ae44576b42d388231f0b50f" candidate_uuid: "3bd6898f790b48d593253180b79c5e40" candidate_term: 2 candidate_status { last_received { term: 1 index: 66 } } ignore_live_leader: true dest_uuid: "5810fd56ce9244438edcd3c0ea23e72c"
I20240501 07:59:27.131024 2537 raft_consensus.cc:3037] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Advancing to term 2
I20240501 07:59:27.132110 2537 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 2 FOLLOWER]: Refusing update from remote peer 3bd6898f790b48d593253180b79c5e40: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 2 index: 68. (index mismatch)
I20240501 07:59:27.132913 4387 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240501 07:59:27.160396 3059 consensus_queue.cc:237] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 112, Committed index: 112, Last appended: 1.112, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 113 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:27.163209 2408 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 112. Preceding OpId from leader: term: 1 index: 113. (index mismatch)
I20240501 07:59:27.163568 2801 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 112. Preceding OpId from leader: term: 1 index: 113. (index mismatch)
I20240501 07:59:27.164034 4017 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 113, Last known committed idx: 112, Time since last communication: 0.000s
I20240501 07:59:27.164202 4309 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 113, Last known committed idx: 112, Time since last communication: 0.000s
I20240501 07:59:27.166589 4325 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 LEADER]: Committing config change with OpId 1.113: config changed from index -1 to 113, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 113 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.168788 2408 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.113: config changed from index -1 to 113, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 113 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.168853 2801 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.113: config changed from index -1 to 113, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 113 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.176487 2670 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 112. Preceding OpId from leader: term: 1 index: 113. (index mismatch)
I20240501 07:59:27.176955 4325 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 113, Last known committed idx: 112, Time since last communication: 0.000s
I20240501 07:59:27.177817 2669 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.113: config changed from index -1 to 113, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 113 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.192589 2538 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 112. Preceding OpId from leader: term: 1 index: 113. (index mismatch)
I20240501 07:59:27.194018 4017 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 113, Last known committed idx: 112, Time since last communication: 0.000s
I20240501 07:59:27.203576 2260 catalog_manager.cc:5484] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index -1 to 113, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "1537338fbeea47ee81f7a210cd6df2f0" committed_config { opid_index: 113 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.204121 2537 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.113: config changed from index -1 to 113, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 113 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.211560 3989 raft_consensus.cc:1050] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2: attempting to promote NON_VOTER e48105f8f6ab476196dd66c4b12808d8 to VOTER
I20240501 07:59:27.225015 2801 consensus_queue.cc:237] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 72, Committed index: 72, Last appended: 2.72, Last appended by leader: 66, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } }
I20240501 07:59:27.226187 2929 consensus_queue.cc:237] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 78, Committed index: 78, Last appended: 1.78, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:27.227290 3989 consensus_queue.cc:237] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 62, Committed index: 62, Last appended: 1.63, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:27.227373 2537 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240501 07:59:27.227723 4195 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 07:59:27.228860 3059 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 07:59:27.228876 3868 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240501 07:59:27.229192 2537 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 07:59:27.229655 4158 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 62, Time since last communication: 0.000s
I20240501 07:59:27.229857 2799 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 07:59:27.229939 3868 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 63. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 07:59:27.229930 4158 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 62, Time since last communication: 0.000s
I20240501 07:59:27.230084 2800 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240501 07:59:27.231557 3989 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 07:59:27.231761 2537 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 2 FOLLOWER]: Refusing update from remote peer 3bd6898f790b48d593253180b79c5e40: Log matching property violated. Preceding OpId in replica: term: 2 index: 72. Preceding OpId from leader: term: 2 index: 73. (index mismatch)
I20240501 07:59:27.231806 3062 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 2 FOLLOWER]: Refusing update from remote peer 3bd6898f790b48d593253180b79c5e40: Log matching property violated. Preceding OpId in replica: term: 2 index: 72. Preceding OpId from leader: term: 2 index: 73. (index mismatch)
I20240501 07:59:27.232142 4388 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 73, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 07:59:27.232461 4387 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 73, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 07:59:27.233811 2408 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 2 FOLLOWER]: Refusing update from remote peer 3bd6898f790b48d593253180b79c5e40: Log matching property violated. Preceding OpId in replica: term: 2 index: 72. Preceding OpId from leader: term: 2 index: 73. (index mismatch)
I20240501 07:59:27.234484 4387 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 73, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 07:59:27.234989 3989 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 62, Time since last communication: 0.000s
I20240501 07:59:27.234946 4383 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [term 2 LEADER]: Committing config change with OpId 2.73: config changed from index 59 to 73, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } } }
I20240501 07:59:27.235491 3989 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 07:59:27.235532 4195 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 64, Last known committed idx: 62, Time since last communication: 0.000s
I20240501 07:59:27.236091 4195 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.79: config changed from index 75 to 79, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.237517 2264 catalog_manager.cc:5484] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 reported cstate change: config changed from index 59 to 73, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New cstate: current_term: 2 leader_uuid: "3bd6898f790b48d593253180b79c5e40" committed_config { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:27.237567 3869 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index 75 to 79, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.238062 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6949f70a1ae44576b42d388231f0b50f with cas_config_opid_index 59: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:27.238334 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a42ea1bbca294f3e9298d895eee087c7 with cas_config_opid_index 75: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:27.240216 2265 catalog_manager.cc:5484] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: config changed from index 75 to 79, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:27.240238 2538 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 5810fd56ce9244438edcd3c0ea23e72c [term 2 FOLLOWER]: Committing config change with OpId 2.73: config changed from index 59 to 73, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } } }
I20240501 07:59:27.241959 2912 tablet_service.cc:1514] Processing DeleteTablet for tablet 6949f70a1ae44576b42d388231f0b50f with delete_type TABLET_DATA_TOMBSTONED (TS e2c508eb3dac4193a1eeba799e5d54f2 not found in new config with opid_index 73) from {username='slave'} at 127.0.0.1:48460
I20240501 07:59:27.239814 3059 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 1537338fbeea47ee81f7a210cd6df2f0 [term 2 FOLLOWER]: Committing config change with OpId 2.73: config changed from index 59 to 73, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } } }
I20240501 07:59:27.243350 3043 tablet_service.cc:1514] Processing DeleteTablet for tablet a42ea1bbca294f3e9298d895eee087c7 with delete_type TABLET_DATA_TOMBSTONED (TS 1537338fbeea47ee81f7a210cd6df2f0 not found in new config with opid_index 79) from {username='slave'} at 127.0.0.1:60186
I20240501 07:59:27.244531 2537 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index 75 to 79, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.246330 4197 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.64: config changed from index 53 to 64, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.248613 2537 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 53 to 64, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.249158 3062 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 53 to 64, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.252440 2264 catalog_manager.cc:5484] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 reported cstate change: config changed from index 53 to 64, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.255172 3869 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 53 to 64, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.258461 2800 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.64: config changed from index 53 to 64, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 64 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.262602 2408 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 85377a976c05408fbb384237a0edb691 [term 2 FOLLOWER]: Committing config change with OpId 2.73: config changed from index 59 to 73, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } } }
I20240501 07:59:27.267360 2801 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index 75 to 79, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.269573 2928 consensus_queue.cc:237] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 64, Committed index: 64, Last appended: 1.64, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:27.272058 2538 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 64. Preceding OpId from leader: term: 1 index: 65. (index mismatch)
I20240501 07:59:27.272425 4196 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 65, Last known committed idx: 64, Time since last communication: 0.000s
I20240501 07:59:27.273521 3870 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 64. Preceding OpId from leader: term: 1 index: 65. (index mismatch)
I20240501 07:59:27.273968 2801 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 64. Preceding OpId from leader: term: 1 index: 65. (index mismatch)
I20240501 07:59:27.275343 3059 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 64. Preceding OpId from leader: term: 1 index: 65. (index mismatch)
I20240501 07:59:27.276324 2670 raft_consensus.cc:1261] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 2 FOLLOWER]: Refusing update from remote peer 3bd6898f790b48d593253180b79c5e40: Log matching property violated. Preceding OpId in replica: term: 2 index: 72. Preceding OpId from leader: term: 2 index: 73. (index mismatch)
I20240501 07:59:27.275808 3989 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 65, Last known committed idx: 64, Time since last communication: 0.000s
I20240501 07:59:27.276961 4389 consensus_queue.cc:1035] T 6949f70a1ae44576b42d388231f0b50f P 3bd6898f790b48d593253180b79c5e40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 73, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 07:59:27.277135 4196 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 65, Last known committed idx: 64, Time since last communication: 0.000s
I20240501 07:59:27.277412 4195 consensus_queue.cc:1035] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 65, Last known committed idx: 64, Time since last communication: 0.000s
I20240501 07:59:27.279211 4195 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.65: config changed from index 64 to 65, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.281767 2537 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.65: config changed from index 64 to 65, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.281767 3060 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.65: config changed from index 64 to 65, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.282006 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 917c88788fbf4089a5df0f271cc478f7 with cas_config_opid_index 64: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:27.282325 2670 raft_consensus.cc:2932] T 6949f70a1ae44576b42d388231f0b50f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 2 FOLLOWER]: Committing config change with OpId 2.73: config changed from index 59 to 73, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { promote: false } } }
I20240501 07:59:27.283900 2264 catalog_manager.cc:5484] T 917c88788fbf4089a5df0f271cc478f7 P 1537338fbeea47ee81f7a210cd6df2f0 reported cstate change: config changed from index 64 to 65, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.287848 2800 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.65: config changed from index 64 to 65, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.290997 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet 917c88788fbf4089a5df0f271cc478f7 with delete_type TABLET_DATA_TOMBSTONED (TS 5dd2fd1701284a6b886ab8d14d1bd8df not found in new config with opid_index 65) from {username='slave'} at 127.0.0.1:44588
I20240501 07:59:27.294234 4403 tablet_replica.cc:330] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0: stopping tablet replica
I20240501 07:59:27.294413 4403 raft_consensus.cc:2229] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:27.301957 3870 raft_consensus.cc:2932] T 917c88788fbf4089a5df0f271cc478f7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.65: config changed from index 64 to 65, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 65 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.311097 2669 raft_consensus.cc:1261] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 62. Preceding OpId from leader: term: 1 index: 64. (index mismatch)
I20240501 07:59:27.311097 2671 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240501 07:59:27.311398 4404 tablet_replica.cc:330] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df: stopping tablet replica
I20240501 07:59:27.312098 4404 raft_consensus.cc:2229] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:27.312263 4404 pending_rounds.cc:62] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df: Trying to abort 1 pending ops.
I20240501 07:59:27.312345 4404 pending_rounds.cc:68] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df: Aborting op as it isn't in flight: id { term: 1 index: 62 } timestamp: 7022798304017526784 op_type: WRITE_OP write_request { tablet_id: "917c88788fbf4089a5df0f271cc478f7" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\2222\251z !\354\304\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""8dcea6e4745d93a1" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7022798303814107136 authz_token { token_data: "\010\206\357\307\261\006\"3\n\005slave\022*\n 0d2f2d6395da492e942ab2c1b19de141\020\001\030\001 \001(\001" signature: "<redacted>""\t;\223\265\035%\031\036\274o\251\345\241\017fHve\326\260_K\020\243\263>?\216\023W)\000U\354\t/X\347_\274+>\212P!\221\247]|\222\346\273<%\231\031\036 \212\233\235b\211\336" signing_key_seq_num: 0 } } request_id { client_id: "64fde3a8b68043daa10e9ff813d56c3e" seq_no: 189 first_incomplete_seq_no: 189 attempt_no: 0 }
I20240501 07:59:27.312608 4404 raft_consensus.cc:2258] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:27.315014 4404 ts_tablet_manager.cc:1875] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:27.317523 4404 ts_tablet_manager.cc:1888] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.62
I20240501 07:59:27.320214 4404 log.cc:1196] T 917c88788fbf4089a5df0f271cc478f7 P 5dd2fd1701284a6b886ab8d14d1bd8df: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/917c88788fbf4089a5df0f271cc478f7
I20240501 07:59:27.320811 2252 catalog_manager.cc:4830] TS 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519): tablet 917c88788fbf4089a5df0f271cc478f7 (table rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]) successfully deleted
I20240501 07:59:27.324020 4195 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 77, Time since last communication: 0.000s
I20240501 07:59:27.329449 2667 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index 75 to 79, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:27.332319 4402 tablet_replica.cc:330] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2: stopping tablet replica
I20240501 07:59:27.332634 4402 raft_consensus.cc:2229] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:27.332808 4403 raft_consensus.cc:2258] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:27.339378 4403 ts_tablet_manager.cc:1875] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20240501 07:59:27.340754 2999 consensus_peers.cc:485] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 -> Peer e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389): Couldn't send request to peer e48105f8f6ab476196dd66c4b12808d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 902e7d877089496689096005187548fe. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:27.341971 4403 ts_tablet_manager.cc:1888] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.78
I20240501 07:59:27.339586 4402 raft_consensus.cc:2258] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:27.343348 4402 ts_tablet_manager.cc:1875] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:27.351437 4403 log.cc:1196] T a42ea1bbca294f3e9298d895eee087c7 P 1537338fbeea47ee81f7a210cd6df2f0: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/a42ea1bbca294f3e9298d895eee087c7
I20240501 07:59:27.352032 2254 catalog_manager.cc:4830] TS 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027): tablet a42ea1bbca294f3e9298d895eee087c7 (table rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]) successfully deleted
I20240501 07:59:27.355526 4402 ts_tablet_manager.cc:1888] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.72
I20240501 07:59:27.366752 4402 log.cc:1196] T 6949f70a1ae44576b42d388231f0b50f P e2c508eb3dac4193a1eeba799e5d54f2: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/6949f70a1ae44576b42d388231f0b50f
I20240501 07:59:27.367414 2253 catalog_manager.cc:4830] TS e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591): tablet 6949f70a1ae44576b42d388231f0b50f (table rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]) successfully deleted
I20240501 07:59:27.396687 2538 consensus_queue.cc:237] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 107, Committed index: 107, Last appended: 1.108, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 109 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:27.401787 2671 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 108. Preceding OpId from leader: term: 1 index: 109. (index mismatch)
I20240501 07:59:27.402251 3935 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 109, Last known committed idx: 107, Time since last communication: 0.000s
I20240501 07:59:27.440248 2929 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 108. Preceding OpId from leader: term: 1 index: 109. (index mismatch)
I20240501 07:59:27.441293 2403 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 108. Preceding OpId from leader: term: 1 index: 109. (index mismatch)
I20240501 07:59:27.441380 2798 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 108. Preceding OpId from leader: term: 1 index: 109. (index mismatch)
I20240501 07:59:27.448288 3935 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 109, Last known committed idx: 107, Time since last communication: 0.000s
I20240501 07:59:27.448570 3935 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 109, Last known committed idx: 107, Time since last communication: 0.000s
I20240501 07:59:27.448784 3935 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 109, Last known committed idx: 107, Time since last communication: 0.000s
I20240501 07:59:27.459640 4407 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 LEADER]: Committing config change with OpId 1.109: config changed from index -1 to 109, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 109 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.466960 2671 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.109: config changed from index -1 to 109, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 109 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.467106 2930 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.109: config changed from index -1 to 109, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 109 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.470172 2800 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.109: config changed from index -1 to 109, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 109 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.470240 2403 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.109: config changed from index -1 to 109, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 109 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.473758 2264 catalog_manager.cc:5484] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 reported cstate change: config changed from index -1 to 109, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "5810fd56ce9244438edcd3c0ea23e72c" committed_config { opid_index: 109 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
W20240501 07:59:27.496361 2475 consensus_peers.cc:485] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c -> Peer e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389): Couldn't send request to peer e48105f8f6ab476196dd66c4b12808d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2fcfbbe5535f45ebad6733432d0f284e. This is attempt 1: this message will repeat every 5th retry.
W20240501 07:59:27.510474 2606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.133:35591, user_credentials={real_user=slave}} blocked reactor thread for 56435.6us
I20240501 07:59:27.631497 2408 consensus_queue.cc:237] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 78, Committed index: 78, Last appended: 1.78, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
W20240501 07:59:27.633157 2344 consensus_peers.cc:485] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 -> Peer e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389): Couldn't send request to peer e48105f8f6ab476196dd66c4b12808d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2a62d5460d7844ebb51f4b33f915bd74. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:27.633898 2670 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240501 07:59:27.634248 4322 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 07:59:27.634580 2539 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240501 07:59:27.634886 2801 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240501 07:59:27.634960 4322 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 07:59:27.635648 4233 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 07:59:27.636847 3063 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 79. (index mismatch)
I20240501 07:59:27.637351 3961 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 79, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 07:59:27.637662 3961 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Committing config change with OpId 1.79: config changed from index -1 to 79, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.639523 2670 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index -1 to 79, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.640426 2264 catalog_manager.cc:5484] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index -1 to 79, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:27.642179 2536 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index -1 to 79, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.643710 3059 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index -1 to 79, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.648332 2800 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.79: config changed from index -1 to 79, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:27.698345 4422 ts_tablet_manager.cc:902] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8: Initiating tablet copy from peer 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027)
I20240501 07:59:27.705495 4422 tablet_copy_client.cc:308] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:44027
I20240501 07:59:27.747385 3084 tablet_copy_service.cc:140] P 1537338fbeea47ee81f7a210cd6df2f0: Received BeginTabletCopySession request for tablet 902e7d877089496689096005187548fe from peer e48105f8f6ab476196dd66c4b12808d8 ({username='slave'} at 127.0.99.135:55817)
I20240501 07:59:27.747519 3084 tablet_copy_service.cc:161] P 1537338fbeea47ee81f7a210cd6df2f0: Beginning new tablet copy session on tablet 902e7d877089496689096005187548fe from peer e48105f8f6ab476196dd66c4b12808d8 at {username='slave'} at 127.0.99.135:55817: session id = e48105f8f6ab476196dd66c4b12808d8-902e7d877089496689096005187548fe
I20240501 07:59:27.748898 3084 tablet_copy_source_session.cc:205] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:27.750674 4422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 902e7d877089496689096005187548fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:27.752985 4422 tablet_copy_client.cc:783] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:27.753129 4422 tablet_copy_client.cc:647] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:27.762678 4422 tablet_copy_client.cc:523] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:27.764334 4422 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8: Bootstrap starting.
I20240501 07:59:27.918900 4422 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=120 overwritten=0 applied=120 ignored=0} inserts{seen=751 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:27.919345 4422 tablet_bootstrap.cc:492] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8: Bootstrap complete.
I20240501 07:59:27.919624 4422 ts_tablet_manager.cc:1367] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8: Time spent bootstrapping tablet: real 0.155s user 0.100s sys 0.006s
I20240501 07:59:27.920186 4422 raft_consensus.cc:348] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 113 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:27.920406 4422 raft_consensus.cc:729] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e48105f8f6ab476196dd66c4b12808d8, State: Initialized, Role: LEARNER
I20240501 07:59:27.920555 4422 consensus_queue.cc:260] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 120, Last appended: 1.120, Last appended by leader: 120, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 113 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:27.921036 4422 ts_tablet_manager.cc:1398] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:27.921548 3084 tablet_copy_service.cc:342] P 1537338fbeea47ee81f7a210cd6df2f0: Request end of tablet copy session e48105f8f6ab476196dd66c4b12808d8-902e7d877089496689096005187548fe received from {username='slave'} at 127.0.99.135:55817
I20240501 07:59:27.921653 3084 tablet_copy_service.cc:433] P 1537338fbeea47ee81f7a210cd6df2f0: ending tablet copy session e48105f8f6ab476196dd66c4b12808d8-902e7d877089496689096005187548fe on tablet 902e7d877089496689096005187548fe with peer e48105f8f6ab476196dd66c4b12808d8
I20240501 07:59:27.959280 4422 ts_tablet_manager.cc:902] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8: Initiating tablet copy from peer 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:27.959704 4422 tablet_copy_client.cc:308] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:32773
I20240501 07:59:27.966305 2560 tablet_copy_service.cc:140] P 5810fd56ce9244438edcd3c0ea23e72c: Received BeginTabletCopySession request for tablet 2fcfbbe5535f45ebad6733432d0f284e from peer e48105f8f6ab476196dd66c4b12808d8 ({username='slave'} at 127.0.99.135:33323)
I20240501 07:59:27.966426 2560 tablet_copy_service.cc:161] P 5810fd56ce9244438edcd3c0ea23e72c: Beginning new tablet copy session on tablet 2fcfbbe5535f45ebad6733432d0f284e from peer e48105f8f6ab476196dd66c4b12808d8 at {username='slave'} at 127.0.99.135:33323: session id = e48105f8f6ab476196dd66c4b12808d8-2fcfbbe5535f45ebad6733432d0f284e
I20240501 07:59:27.967783 2560 tablet_copy_source_session.cc:205] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:27.969153 4422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcfbbe5535f45ebad6733432d0f284e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:27.971325 4422 tablet_copy_client.cc:783] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:27.971469 4422 tablet_copy_client.cc:647] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:27.977360 4422 tablet_copy_client.cc:523] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:27.978513 4422 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8: Bootstrap starting.
I20240501 07:59:28.088111 3868 raft_consensus.cc:1203] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.112->[1.113-1.126] Dedup: 1.120->[1.121-1.126]
I20240501 07:59:28.114567 4326 raft_consensus.cc:1050] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0: attempting to promote NON_VOTER e48105f8f6ab476196dd66c4b12808d8 to VOTER
I20240501 07:59:28.115128 4326 consensus_queue.cc:237] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 126, Committed index: 126, Last appended: 1.126, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 127 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:28.116657 2540 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 126. Preceding OpId from leader: term: 1 index: 127. (index mismatch)
I20240501 07:59:28.116906 3868 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 126. Preceding OpId from leader: term: 1 index: 127. (index mismatch)
I20240501 07:59:28.117022 2404 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 126. Preceding OpId from leader: term: 1 index: 127. (index mismatch)
I20240501 07:59:28.117465 2669 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 126. Preceding OpId from leader: term: 1 index: 127. (index mismatch)
I20240501 07:59:28.117511 4325 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 127, Last known committed idx: 126, Time since last communication: 0.000s
I20240501 07:59:28.117656 4324 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 127, Last known committed idx: 126, Time since last communication: 0.000s
I20240501 07:59:28.117919 4324 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 127, Last known committed idx: 126, Time since last communication: 0.000s
I20240501 07:59:28.118096 4017 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 127, Last known committed idx: 126, Time since last communication: 0.000s
I20240501 07:59:28.118803 2797 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 126. Preceding OpId from leader: term: 1 index: 127. (index mismatch)
I20240501 07:59:28.120265 4017 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 LEADER]: Committing config change with OpId 1.127: config changed from index 113 to 127, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 127 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.120548 2539 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.127: config changed from index 113 to 127, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 127 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.124290 2264 catalog_manager.cc:5484] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c reported cstate change: config changed from index 113 to 127, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1537338fbeea47ee81f7a210cd6df2f0" committed_config { opid_index: 127 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.125622 2669 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.127: config changed from index 113 to 127, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 127 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.125769 4325 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 127, Last known committed idx: 126, Time since last communication: 0.000s
I20240501 07:59:28.127450 2408 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.127: config changed from index 113 to 127, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 127 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.129438 2798 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.127: config changed from index 113 to 127, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 127 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.131429 3868 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.127: config changed from index 113 to 127, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 127 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.142499 4465 ts_tablet_manager.cc:902] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8: Initiating tablet copy from peer 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933)
I20240501 07:59:28.148914 4465 tablet_copy_client.cc:308] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:42933
I20240501 07:59:28.161039 2429 tablet_copy_service.cc:140] P 85377a976c05408fbb384237a0edb691: Received BeginTabletCopySession request for tablet 2a62d5460d7844ebb51f4b33f915bd74 from peer e48105f8f6ab476196dd66c4b12808d8 ({username='slave'} at 127.0.99.135:52723)
I20240501 07:59:28.161149 2429 tablet_copy_service.cc:161] P 85377a976c05408fbb384237a0edb691: Beginning new tablet copy session on tablet 2a62d5460d7844ebb51f4b33f915bd74 from peer e48105f8f6ab476196dd66c4b12808d8 at {username='slave'} at 127.0.99.135:52723: session id = e48105f8f6ab476196dd66c4b12808d8-2a62d5460d7844ebb51f4b33f915bd74
I20240501 07:59:28.162056 2429 tablet_copy_source_session.cc:205] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:28.163040 4465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a62d5460d7844ebb51f4b33f915bd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:28.231503 4422 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=122 overwritten=0 applied=122 ignored=0} inserts{seen=807 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:28.233059 4422 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8: Bootstrap complete.
I20240501 07:59:28.233318 4422 ts_tablet_manager.cc:1367] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8: Time spent bootstrapping tablet: real 0.255s user 0.094s sys 0.015s
I20240501 07:59:28.233803 4422 raft_consensus.cc:348] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 109 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:28.234020 4422 raft_consensus.cc:729] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e48105f8f6ab476196dd66c4b12808d8, State: Initialized, Role: LEARNER
I20240501 07:59:28.234191 4422 consensus_queue.cc:260] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 122, Last appended: 1.122, Last appended by leader: 122, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 109 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:28.236181 4422 ts_tablet_manager.cc:1398] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:28.239246 2560 tablet_copy_service.cc:342] P 5810fd56ce9244438edcd3c0ea23e72c: Request end of tablet copy session e48105f8f6ab476196dd66c4b12808d8-2fcfbbe5535f45ebad6733432d0f284e received from {username='slave'} at 127.0.99.135:33323
I20240501 07:59:28.239342 2560 tablet_copy_service.cc:433] P 5810fd56ce9244438edcd3c0ea23e72c: ending tablet copy session e48105f8f6ab476196dd66c4b12808d8-2fcfbbe5535f45ebad6733432d0f284e on tablet 2fcfbbe5535f45ebad6733432d0f284e with peer e48105f8f6ab476196dd66c4b12808d8
I20240501 07:59:28.280488 4465 tablet_copy_client.cc:783] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:28.280685 4465 tablet_copy_client.cc:647] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:28.284721 4465 tablet_copy_client.cc:523] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:28.286914 4465 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap starting.
I20240501 07:59:28.303202 2929 consensus_queue.cc:237] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:28.305253 2535 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:28.305749 3995 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 07:59:28.308823 3063 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:28.310956 3995 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 07:59:28.312667 2669 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:28.314576 4401 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 07:59:28.320312 4195 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:28.323271 2538 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:28.325922 3063 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:28.326469 2261 catalog_manager.cc:5484] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c reported cstate change: config changed from index -1 to 2, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:28.344463 2408 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:28.347954 3995 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 07:59:28.348109 2669 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
W20240501 07:59:28.348759 2868 consensus_peers.cc:485] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 -> Peer e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389): Couldn't send request to peer e48105f8f6ab476196dd66c4b12808d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cb4496116793436f849ea95bcc6fc6ba. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:28.352782 2404 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:28.360320 3868 raft_consensus.cc:1203] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.108->[1.109-1.128] Dedup: 1.122->[1.123-1.128]
W20240501 07:59:28.434715 4017 consensus_queue.cc:1159] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5810fd56ce9244438edcd3c0ea23e72c" responder_term: 1 status { last_received { term: 1 index: 129 } last_committed_idx: 128 last_received_current_leader { term: 1 index: 129 } }
I20240501 07:59:28.435613 3060 consensus_queue.cc:237] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 128, Committed index: 128, Last appended: 1.129, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:28.437058 2670 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 129. Preceding OpId from leader: term: 1 index: 130. (index mismatch)
I20240501 07:59:28.437523 2407 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 129. Preceding OpId from leader: term: 1 index: 130. (index mismatch)
I20240501 07:59:28.437783 4017 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 130, Last known committed idx: 128, Time since last communication: 0.000s
I20240501 07:59:28.437904 4326 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 130, Last known committed idx: 128, Time since last communication: 0.000s
I20240501 07:59:28.440786 4017 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 LEADER]: Committing config change with OpId 1.130: config changed from index 127 to 130, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.442836 2254 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 902e7d877089496689096005187548fe with cas_config_opid_index 127: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:28.443691 3869 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 129. Preceding OpId from leader: term: 1 index: 130. (index mismatch)
I20240501 07:59:28.444561 2407 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.130: config changed from index 127 to 130, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.446460 4325 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 130, Last known committed idx: 128, Time since last communication: 0.000s
I20240501 07:59:28.447660 2261 catalog_manager.cc:5484] T 902e7d877089496689096005187548fe P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index 127 to 130, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "1537338fbeea47ee81f7a210cd6df2f0" committed_config { opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.448210 3869 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.130: config changed from index 127 to 130, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.450234 2667 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.130: config changed from index 127 to 130, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.451028 2520 tablet_service.cc:1514] Processing DeleteTablet for tablet 902e7d877089496689096005187548fe with delete_type TABLET_DATA_TOMBSTONED (TS 5810fd56ce9244438edcd3c0ea23e72c not found in new config with opid_index 130) from {username='slave'} at 127.0.0.1:56996
I20240501 07:59:28.460289 2798 raft_consensus.cc:1261] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 127. Preceding OpId from leader: term: 1 index: 130. (index mismatch)
I20240501 07:59:28.503906 4502 tablet_replica.cc:330] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c: stopping tablet replica
I20240501 07:59:28.505010 4502 raft_consensus.cc:2229] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:28.505544 4502 pending_rounds.cc:62] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c: Trying to abort 1 pending ops.
I20240501 07:59:28.505654 4502 pending_rounds.cc:68] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c: Aborting op as it isn't in flight: id { term: 1 index: 129 } timestamp: 7022798309084979200 op_type: WRITE_OP write_request { tablet_id: "902e7d877089496689096005187548fe" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000oW\355^\361\000\205\266\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""6b072e3bc8d8b4c4" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7022798309060644864 authz_token { token_data: "\010\206\357\307\261\006\"3\n\005slave\022*\n b69d178630d34f13b7c00a0f739d1d9e\020\001\030\001 \001(\001" signature: "<redacted>""I\023\341Q\215\312\265\367\233$=6 \334\370\341Upb\355\330\350nB\3376\227\212\223\333-\371\224Sve\311OQ\216S\271\243\217\027Q\177\312`=\250\341\030V\323%\311\202\235\025]A\304U" signing_key_seq_num: 0 } } request_id { client_id: "e3af9f411f3e4163b2886849f24be5f0" seq_no: 376 first_incomplete_seq_no: 376 attempt_no: 0 }
I20240501 07:59:28.507321 4502 raft_consensus.cc:2258] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:28.525998 4502 ts_tablet_manager.cc:1875] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:28.539911 4502 ts_tablet_manager.cc:1888] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.129
I20240501 07:59:28.540115 4502 log.cc:1196] T 902e7d877089496689096005187548fe P 5810fd56ce9244438edcd3c0ea23e72c: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/902e7d877089496689096005187548fe
I20240501 07:59:28.540787 3935 raft_consensus.cc:1050] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c: attempting to promote NON_VOTER e48105f8f6ab476196dd66c4b12808d8 to VOTER
I20240501 07:59:28.541111 2251 catalog_manager.cc:4830] TS 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773): tablet 902e7d877089496689096005187548fe (table rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]) successfully deleted
I20240501 07:59:28.541334 3935 consensus_queue.cc:237] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 131, Committed index: 131, Last appended: 1.131, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 132 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:28.543288 3869 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 131. Preceding OpId from leader: term: 1 index: 132. (index mismatch)
I20240501 07:59:28.543642 3937 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 132, Last known committed idx: 131, Time since last communication: 0.000s
I20240501 07:59:28.544199 2798 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 131. Preceding OpId from leader: term: 1 index: 132. (index mismatch)
I20240501 07:59:28.544551 2928 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 131. Preceding OpId from leader: term: 1 index: 132. (index mismatch)
I20240501 07:59:28.544955 3937 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 132, Last known committed idx: 131, Time since last communication: 0.000s
I20240501 07:59:28.545233 2670 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 131. Preceding OpId from leader: term: 1 index: 132. (index mismatch)
I20240501 07:59:28.545586 3937 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 132, Last known committed idx: 131, Time since last communication: 0.000s
I20240501 07:59:28.547974 3947 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 LEADER]: Committing config change with OpId 1.132: config changed from index 109 to 132, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 132 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.548492 2671 consensus_queue.cc:237] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 96, Committed index: 96, Last appended: 1.96, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 97 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:28.548642 2928 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.132: config changed from index 109 to 132, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 132 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.550346 2798 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.132: config changed from index 109 to 132, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 132 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.550757 2264 catalog_manager.cc:5484] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c reported cstate change: config changed from index 109 to 132, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5810fd56ce9244438edcd3c0ea23e72c" committed_config { opid_index: 132 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:28.551728 4407 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 132, Last known committed idx: 131, Time since last communication: 0.000s
I20240501 07:59:28.552772 2670 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.132: config changed from index 109 to 132, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 132 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.553750 3868 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.132: config changed from index 109 to 132, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 132 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.561247 2538 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 96. Preceding OpId from leader: term: 1 index: 97. (index mismatch)
I20240501 07:59:28.561384 3063 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 96. Preceding OpId from leader: term: 1 index: 97. (index mismatch)
I20240501 07:59:28.561643 4020 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 97, Last known committed idx: 96, Time since last communication: 0.000s
I20240501 07:59:28.561892 4020 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 97, Last known committed idx: 96, Time since last communication: 0.000s
I20240501 07:59:28.563359 4020 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.97: config changed from index -1 to 97, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 97 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:28.565310 2538 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.97: config changed from index -1 to 97, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 97 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:28.567677 2261 catalog_manager.cc:5484] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c reported cstate change: config changed from index -1 to 97, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 97 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:28.568884 3059 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.97: config changed from index -1 to 97, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 97 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
W20240501 07:59:28.572024 2606 consensus_peers.cc:485] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df -> Peer e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389): Couldn't send request to peer e48105f8f6ab476196dd66c4b12808d8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 382e9b7922f849f980b80ded29e99075. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:28.580929 4465 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=91 overwritten=0 applied=91 ignored=0} inserts{seen=722 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:28.585397 4465 tablet_bootstrap.cc:492] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap complete.
I20240501 07:59:28.585852 4465 ts_tablet_manager.cc:1367] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8: Time spent bootstrapping tablet: real 0.299s user 0.075s sys 0.012s
I20240501 07:59:28.586298 4465 raft_consensus.cc:348] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:28.586426 4465 raft_consensus.cc:729] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e48105f8f6ab476196dd66c4b12808d8, State: Initialized, Role: LEARNER
I20240501 07:59:28.588724 4465 consensus_queue.cc:260] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 91, Last appended: 1.91, Last appended by leader: 91, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 79 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:28.588783 4434 consensus_queue.cc:1035] T 902e7d877089496689096005187548fe P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 130, Last known committed idx: 127, Time since last communication: 0.000s
I20240501 07:59:28.589558 4465 ts_tablet_manager.cc:1398] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240501 07:59:28.590565 2429 tablet_copy_service.cc:342] P 85377a976c05408fbb384237a0edb691: Request end of tablet copy session e48105f8f6ab476196dd66c4b12808d8-2a62d5460d7844ebb51f4b33f915bd74 received from {username='slave'} at 127.0.99.135:52723
I20240501 07:59:28.590632 2429 tablet_copy_service.cc:433] P 85377a976c05408fbb384237a0edb691: ending tablet copy session e48105f8f6ab476196dd66c4b12808d8-2a62d5460d7844ebb51f4b33f915bd74 on tablet 2a62d5460d7844ebb51f4b33f915bd74 with peer e48105f8f6ab476196dd66c4b12808d8
I20240501 07:59:28.594031 2408 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 94. Preceding OpId from leader: term: 1 index: 97. (index mismatch)
I20240501 07:59:28.611794 2800 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 94. Preceding OpId from leader: term: 1 index: 97. (index mismatch)
I20240501 07:59:28.617395 2536 consensus_queue.cc:237] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 133, Committed index: 133, Last appended: 1.133, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 134 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:28.618971 2671 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 133. Preceding OpId from leader: term: 1 index: 134. (index mismatch)
I20240501 07:59:28.619612 3934 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 134, Last known committed idx: 133, Time since last communication: 0.000s
I20240501 07:59:28.621847 2404 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 131. Preceding OpId from leader: term: 1 index: 132. (index mismatch)
I20240501 07:59:28.622700 2798 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 133. Preceding OpId from leader: term: 1 index: 134. (index mismatch)
I20240501 07:59:28.623155 2928 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 133. Preceding OpId from leader: term: 1 index: 134. (index mismatch)
I20240501 07:59:28.623559 3934 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 134, Last known committed idx: 132, Time since last communication: 0.000s
I20240501 07:59:28.623821 3934 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 134, Last known committed idx: 133, Time since last communication: 0.000s
I20240501 07:59:28.626156 3868 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 133. Preceding OpId from leader: term: 1 index: 134. (index mismatch)
I20240501 07:59:28.630146 3935 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 134, Last known committed idx: 133, Time since last communication: 0.000s
I20240501 07:59:28.630887 3935 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 LEADER]: Committing config change with OpId 1.134: config changed from index 132 to 134, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 134 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.633808 2928 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.134: config changed from index 132 to 134, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 134 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.635851 2799 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.134: config changed from index 132 to 134, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 134 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.637589 2251 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2fcfbbe5535f45ebad6733432d0f284e with cas_config_opid_index 132: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:28.638526 2667 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.134: config changed from index 132 to 134, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 134 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.638953 2264 catalog_manager.cc:5484] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: config changed from index 132 to 134, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "5810fd56ce9244438edcd3c0ea23e72c" committed_config { opid_index: 134 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.639775 2800 raft_consensus.cc:1203] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.127->[1.128-1.132] Dedup: 1.128->[1.129-1.132]
I20240501 07:59:28.640492 4174 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 97, Last known committed idx: 94, Time since last communication: 0.000s
I20240501 07:59:28.640976 2800 raft_consensus.cc:2932] T 902e7d877089496689096005187548fe P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.130: config changed from index 127 to 130, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.641819 4174 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 97, Last known committed idx: 94, Time since last communication: 0.000s
I20240501 07:59:28.644703 2387 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fcfbbe5535f45ebad6733432d0f284e with delete_type TABLET_DATA_TOMBSTONED (TS 85377a976c05408fbb384237a0edb691 not found in new config with opid_index 134) from {username='slave'} at 127.0.0.1:48528
I20240501 07:59:28.649191 3868 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.134: config changed from index 132 to 134, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 134 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:28.658569 2408 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.97: config changed from index -1 to 97, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 97 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:28.663756 2797 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.97: config changed from index -1 to 97, NON_VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) added. New config: { opid_index: 97 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } } }
I20240501 07:59:28.676373 4514 tablet_replica.cc:330] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691: stopping tablet replica
I20240501 07:59:28.676529 4514 raft_consensus.cc:2229] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:28.676821 4514 raft_consensus.cc:2258] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:28.677637 4514 ts_tablet_manager.cc:1875] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:28.681339 4514 ts_tablet_manager.cc:1888] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.131
I20240501 07:59:28.681412 4514 log.cc:1196] T 2fcfbbe5535f45ebad6733432d0f284e P 85377a976c05408fbb384237a0edb691: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/2fcfbbe5535f45ebad6733432d0f284e
I20240501 07:59:28.682691 2253 catalog_manager.cc:4830] TS 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933): tablet 2fcfbbe5535f45ebad6733432d0f284e (table rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]) successfully deleted
I20240501 07:59:28.869202 4465 ts_tablet_manager.cc:902] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8: Initiating tablet copy from peer e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591)
I20240501 07:59:28.869642 4465 tablet_copy_client.cc:308] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:35591
I20240501 07:59:28.870203 2953 tablet_copy_service.cc:140] P e2c508eb3dac4193a1eeba799e5d54f2: Received BeginTabletCopySession request for tablet cb4496116793436f849ea95bcc6fc6ba from peer e48105f8f6ab476196dd66c4b12808d8 ({username='slave'} at 127.0.99.135:50307)
I20240501 07:59:28.870306 2953 tablet_copy_service.cc:161] P e2c508eb3dac4193a1eeba799e5d54f2: Beginning new tablet copy session on tablet cb4496116793436f849ea95bcc6fc6ba from peer e48105f8f6ab476196dd66c4b12808d8 at {username='slave'} at 127.0.99.135:50307: session id = e48105f8f6ab476196dd66c4b12808d8-cb4496116793436f849ea95bcc6fc6ba
I20240501 07:59:28.871198 2953 tablet_copy_source_session.cc:205] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:28.933508 3866 raft_consensus.cc:1203] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.78->[1.79-1.105] Dedup: 1.91->[1.92-1.105]
I20240501 07:59:28.976673 4465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4496116793436f849ea95bcc6fc6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:28.978744 4465 tablet_copy_client.cc:783] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:28.978901 4465 tablet_copy_client.cc:647] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:29.025719 4519 raft_consensus.cc:1050] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691: attempting to promote NON_VOTER e48105f8f6ab476196dd66c4b12808d8 to VOTER
I20240501 07:59:29.026160 4519 consensus_queue.cc:237] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 106, Committed index: 106, Last appended: 1.107, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:29.027738 2798 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 107. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240501 07:59:29.028326 2667 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 107. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240501 07:59:29.028573 3867 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 107. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240501 07:59:29.029148 4148 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 106, Time since last communication: 0.000s
I20240501 07:59:29.029525 4148 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 106, Time since last communication: 0.000s
I20240501 07:59:29.029903 4322 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 106, Time since last communication: 0.000s
I20240501 07:59:29.032424 2539 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 107. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240501 07:59:29.033010 4519 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 106, Time since last communication: 0.000s
I20240501 07:59:29.033485 4520 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Committing config change with OpId 1.108: config changed from index 79 to 108, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.033953 2540 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 79 to 108, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.036063 2667 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 79 to 108, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.036393 3059 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 107. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240501 07:59:29.036737 2261 catalog_manager.cc:5484] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index 79 to 108, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:29.038146 3867 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 79 to 108, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.040558 4465 tablet_copy_client.cc:523] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:29.041880 4465 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8: Bootstrap starting.
I20240501 07:59:29.045830 4518 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 106, Time since last communication: 0.000s
I20240501 07:59:29.040272 2797 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 79 to 108, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.056106 3063 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 79 to 108, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.069739 2408 consensus_queue.cc:237] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 109, Committed index: 109, Last appended: 1.109, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:29.073083 2670 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 109. Preceding OpId from leader: term: 1 index: 110. (index mismatch)
I20240501 07:59:29.073563 4519 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 110, Last known committed idx: 109, Time since last communication: 0.000s
I20240501 07:59:29.073833 3867 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 109. Preceding OpId from leader: term: 1 index: 110. (index mismatch)
I20240501 07:59:29.074331 4519 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 110, Last known committed idx: 109, Time since last communication: 0.000s
I20240501 07:59:29.077947 2798 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 109. Preceding OpId from leader: term: 1 index: 110. (index mismatch)
I20240501 07:59:29.078727 4518 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 110, Last known committed idx: 109, Time since last communication: 0.000s
I20240501 07:59:29.083290 4233 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Committing config change with OpId 1.110: config changed from index 108 to 110, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.085515 2667 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.110: config changed from index 108 to 110, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.086110 2261 catalog_manager.cc:5484] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index 108 to 110, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:29.088253 4465 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:29.088486 4465 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8: Bootstrap complete.
I20240501 07:59:29.088634 4465 ts_tablet_manager.cc:1367] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.000s
I20240501 07:59:29.089092 4465 raft_consensus.cc:348] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:29.089208 4465 raft_consensus.cc:729] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e48105f8f6ab476196dd66c4b12808d8, State: Initialized, Role: LEARNER
I20240501 07:59:29.089874 2520 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a62d5460d7844ebb51f4b33f915bd74 with delete_type TABLET_DATA_TOMBSTONED (TS 5810fd56ce9244438edcd3c0ea23e72c not found in new config with opid_index 110) from {username='slave'} at 127.0.0.1:56996
I20240501 07:59:29.089308 4465 consensus_queue.cc:260] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:29.097465 3059 raft_consensus.cc:1261] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 109. Preceding OpId from leader: term: 1 index: 110. (index mismatch)
I20240501 07:59:29.097836 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2a62d5460d7844ebb51f4b33f915bd74 with cas_config_opid_index 108: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:29.099182 4465 ts_tablet_manager.cc:1398] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20240501 07:59:29.099807 3867 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.110: config changed from index 108 to 110, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.101403 2953 tablet_copy_service.cc:342] P e2c508eb3dac4193a1eeba799e5d54f2: Request end of tablet copy session e48105f8f6ab476196dd66c4b12808d8-cb4496116793436f849ea95bcc6fc6ba received from {username='slave'} at 127.0.99.135:50307
I20240501 07:59:29.101488 2953 tablet_copy_service.cc:433] P e2c508eb3dac4193a1eeba799e5d54f2: ending tablet copy session e48105f8f6ab476196dd66c4b12808d8-cb4496116793436f849ea95bcc6fc6ba on tablet cb4496116793436f849ea95bcc6fc6ba with peer e48105f8f6ab476196dd66c4b12808d8
I20240501 07:59:29.106611 2797 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.110: config changed from index 108 to 110, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.114528 4233 consensus_queue.cc:1035] T 2a62d5460d7844ebb51f4b33f915bd74 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 110, Last known committed idx: 109, Time since last communication: 0.000s
I20240501 07:59:29.120334 3062 raft_consensus.cc:2932] T 2a62d5460d7844ebb51f4b33f915bd74 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.110: config changed from index 108 to 110, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.148424 4541 tablet_replica.cc:330] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c: stopping tablet replica
I20240501 07:59:29.149899 4541 raft_consensus.cc:2229] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:29.150195 4541 raft_consensus.cc:2258] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:29.155633 4541 ts_tablet_manager.cc:1875] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:29.161185 4465 ts_tablet_manager.cc:902] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8: Initiating tablet copy from peer 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519)
I20240501 07:59:29.161679 4465 tablet_copy_client.cc:308] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:44519
I20240501 07:59:29.172411 2691 tablet_copy_service.cc:140] P 5dd2fd1701284a6b886ab8d14d1bd8df: Received BeginTabletCopySession request for tablet 382e9b7922f849f980b80ded29e99075 from peer e48105f8f6ab476196dd66c4b12808d8 ({username='slave'} at 127.0.99.135:36749)
I20240501 07:59:29.172516 2691 tablet_copy_service.cc:161] P 5dd2fd1701284a6b886ab8d14d1bd8df: Beginning new tablet copy session on tablet 382e9b7922f849f980b80ded29e99075 from peer e48105f8f6ab476196dd66c4b12808d8 at {username='slave'} at 127.0.99.135:36749: session id = e48105f8f6ab476196dd66c4b12808d8-382e9b7922f849f980b80ded29e99075
I20240501 07:59:29.173389 2691 tablet_copy_source_session.cc:205] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:29.174904 4541 ts_tablet_manager.cc:1888] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.109
I20240501 07:59:29.175909 4541 log.cc:1196] T 2a62d5460d7844ebb51f4b33f915bd74 P 5810fd56ce9244438edcd3c0ea23e72c: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/2a62d5460d7844ebb51f4b33f915bd74
I20240501 07:59:29.185187 4465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 382e9b7922f849f980b80ded29e99075. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:29.185492 2251 catalog_manager.cc:4830] TS 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773): tablet 2a62d5460d7844ebb51f4b33f915bd74 (table rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]) successfully deleted
I20240501 07:59:29.188697 4465 tablet_copy_client.cc:783] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:29.188871 4465 tablet_copy_client.cc:647] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:29.264510 4465 tablet_copy_client.cc:523] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:29.269734 4465 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap starting.
I20240501 07:59:29.280514 3866 raft_consensus.cc:1203] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 07:59:29.618218 4465 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=104 overwritten=0 applied=104 ignored=0} inserts{seen=735 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:29.619138 4465 tablet_bootstrap.cc:492] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8: Bootstrap complete.
I20240501 07:59:29.619396 4465 ts_tablet_manager.cc:1367] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8: Time spent bootstrapping tablet: real 0.350s user 0.084s sys 0.012s
I20240501 07:59:29.619913 4465 raft_consensus.cc:348] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 97 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:29.620162 4465 raft_consensus.cc:729] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e48105f8f6ab476196dd66c4b12808d8, State: Initialized, Role: LEARNER
I20240501 07:59:29.621443 4465 consensus_queue.cc:260] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 104, Last appended: 1.104, Last appended by leader: 104, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 97 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: true } }
I20240501 07:59:29.624024 4465 ts_tablet_manager.cc:1398] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240501 07:59:29.664755 2691 tablet_copy_service.cc:342] P 5dd2fd1701284a6b886ab8d14d1bd8df: Request end of tablet copy session e48105f8f6ab476196dd66c4b12808d8-382e9b7922f849f980b80ded29e99075 received from {username='slave'} at 127.0.99.135:36749
I20240501 07:59:29.664858 2691 tablet_copy_service.cc:433] P 5dd2fd1701284a6b886ab8d14d1bd8df: ending tablet copy session e48105f8f6ab476196dd66c4b12808d8-382e9b7922f849f980b80ded29e99075 on tablet 382e9b7922f849f980b80ded29e99075 with peer e48105f8f6ab476196dd66c4b12808d8
I20240501 07:59:29.804155 4158 raft_consensus.cc:1050] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2: attempting to promote NON_VOTER e48105f8f6ab476196dd66c4b12808d8 to VOTER
I20240501 07:59:29.804589 4158 consensus_queue.cc:237] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:29.806456 2538 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 07:59:29.806869 3989 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 07:59:29.809090 3059 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 07:59:29.811599 3989 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 07:59:29.820189 2670 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 07:59:29.822930 3870 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 07:59:29.824568 2404 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 07:59:29.843222 4585 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 07:59:29.848464 4585 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 07:59:29.851861 4585 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 07:59:29.868110 3870 raft_consensus.cc:1203] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.96->[1.97-1.113] Dedup: 1.104->[1.105-1.113]
I20240501 07:59:29.889196 4158 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.891263 3059 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.891834 2537 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.895411 2261 catalog_manager.cc:5484] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c reported cstate change: config changed from index 2 to 3, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.896462 2403 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.933931 2670 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.950032 3867 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:29.995725 4018 raft_consensus.cc:1050] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df: attempting to promote NON_VOTER e48105f8f6ab476196dd66c4b12808d8 to VOTER
I20240501 07:59:29.996290 4018 consensus_queue.cc:237] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 114, Committed index: 114, Last appended: 1.114, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:29.997876 2540 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240501 07:59:29.999239 4298 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 114, Time since last communication: 0.000s
I20240501 07:59:30.000228 3060 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240501 07:59:30.001121 4018 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 114, Time since last communication: 0.000s
W20240501 07:59:30.044871 2606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.129:42933, user_credentials={real_user=slave}} blocked reactor thread for 46188.7us
I20240501 07:59:30.045430 3868 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8 [term 1 LEARNER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240501 07:59:30.045799 2931 consensus_queue.cc:237] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:30.047456 2404 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240501 07:59:30.048127 3059 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:30.048722 3995 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:30.050194 2799 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240501 07:59:30.051287 2535 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:30.052536 4195 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:30.052580 2404 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:30.053951 3995 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:30.055107 3870 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:30.055461 3995 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:30.058948 4551 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 113, Time since last communication: 0.000s
I20240501 07:59:30.059356 3995 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.061542 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet cb4496116793436f849ea95bcc6fc6ba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:30.062254 2538 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.063982 4551 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 114, Time since last communication: 0.000s
I20240501 07:59:30.065253 3062 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.065356 2258 catalog_manager.cc:5484] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c reported cstate change: config changed from index 3 to 4, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.069594 3870 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.072675 2404 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.083447 4551 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 114, Time since last communication: 0.000s
I20240501 07:59:30.090289 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet cb4496116793436f849ea95bcc6fc6ba with delete_type TABLET_DATA_TOMBSTONED (TS 5dd2fd1701284a6b886ab8d14d1bd8df not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44588
I20240501 07:59:30.108196 4551 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.115: config changed from index 97 to 115, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.110688 3059 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index 97 to 115, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.114702 2260 catalog_manager.cc:5484] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index 97 to 115, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:30.121635 3868 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index 97 to 115, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.123478 2538 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index 97 to 115, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.135725 2408 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index 97 to 115, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.134351 4605 tablet_replica.cc:330] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df: stopping tablet replica
I20240501 07:59:30.144012 4605 raft_consensus.cc:2229] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:30.144259 4605 raft_consensus.cc:2258] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:30.149847 2669 consensus_queue.cc:237] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 116, Committed index: 116, Last appended: 1.116, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } }
I20240501 07:59:30.152629 4605 ts_tablet_manager.cc:1875] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:30.153645 3060 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 116. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240501 07:59:30.154417 4018 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 116, Time since last communication: 0.000s
I20240501 07:59:30.158619 3870 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 116. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240501 07:59:30.160562 2408 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 116. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240501 07:59:30.162613 4298 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 116, Time since last communication: 0.000s
I20240501 07:59:30.165944 4298 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 116, Time since last communication: 0.000s
I20240501 07:59:30.167891 3740 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.117: config changed from index 115 to 117, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.173734 2407 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 115 to 117, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.175066 3870 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 115 to 117, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.176765 2266 catalog_manager.cc:5484] T 382e9b7922f849f980b80ded29e99075 P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index 115 to 117, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.185128 2520 tablet_service.cc:1514] Processing DeleteTablet for tablet 382e9b7922f849f980b80ded29e99075 with delete_type TABLET_DATA_TOMBSTONED (TS 5810fd56ce9244438edcd3c0ea23e72c not found in new config with opid_index 117) from {username='slave'} at 127.0.0.1:56996
I20240501 07:59:30.185545 4610 tablet_replica.cc:330] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c: stopping tablet replica
I20240501 07:59:30.185719 4610 raft_consensus.cc:2229] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:30.186125 4610 raft_consensus.cc:2258] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:30.189368 2252 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 382e9b7922f849f980b80ded29e99075 with cas_config_opid_index 115: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:30.194449 3059 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 115 to 117, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.190075 4610 ts_tablet_manager.cc:1875] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:30.218705 4610 ts_tablet_manager.cc:1888] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.116
I20240501 07:59:30.218888 4610 log.cc:1196] T 382e9b7922f849f980b80ded29e99075 P 5810fd56ce9244438edcd3c0ea23e72c: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/382e9b7922f849f980b80ded29e99075
I20240501 07:59:30.233167 2251 catalog_manager.cc:4830] TS 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773): tablet 382e9b7922f849f980b80ded29e99075 (table rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]) successfully deleted
I20240501 07:59:30.251436 4605 ts_tablet_manager.cc:1888] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 07:59:30.252341 4605 log.cc:1196] T cb4496116793436f849ea95bcc6fc6ba P 5dd2fd1701284a6b886ab8d14d1bd8df: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb4496116793436f849ea95bcc6fc6ba
I20240501 07:59:30.254292 2252 catalog_manager.cc:4830] TS 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519): tablet cb4496116793436f849ea95bcc6fc6ba (table TestTable [id=8e1fca9f5d8e40cd8da5d523ca3d4e09]) successfully deleted
I20240501 07:59:30.281993 2797 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index 97 to 115, e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
I20240501 07:59:30.288200 2797 raft_consensus.cc:1261] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 116. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240501 07:59:30.296772 4018 consensus_queue.cc:1035] T 382e9b7922f849f980b80ded29e99075 P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 116, Time since last communication: 0.000s
I20240501 07:59:30.394917 2798 raft_consensus.cc:2932] T 382e9b7922f849f980b80ded29e99075 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 115 to 117, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } }
W20240501 07:59:30.593070 2997 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.130:32773, user_credentials={real_user=slave}} blocked reactor thread for 99664us
W20240501 07:59:31.190383 2606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.135:37389, user_credentials={real_user=slave}} blocked reactor thread for 61747.6us
I20240501 07:59:31.227656 3043 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 07:59:31.296880 2912 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:31.377954 2649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 07:59:31.385813 2520 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:31.423642 2387 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:31.442440 2782 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:31.488435 3850 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:32.254647 2782 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:32.354398 2649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 07:59:32.375649 2520 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:32.390345 3850 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:32.394534 3043 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 07:59:32.448271 2912 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:32.509951 2387 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
W20240501 07:59:33.014847 2606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.135:37389, user_credentials={real_user=slave}} blocked reactor thread for 83459.1us
W20240501 07:59:33.383543 2869 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.134:44027, user_credentials={real_user=slave}} blocked reactor thread for 83404.3us
I20240501 07:59:34.106160 2649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 07:59:34.186553 2520 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:34.191542 2912 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:34.191766 3850 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20240501 07:59:34.243558 2345 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.134:44027, user_credentials={real_user=slave}} blocked reactor thread for 54882us
I20240501 07:59:34.330554 3043 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 07:59:34.341404 2782 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:34.463306 2387 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
W20240501 07:59:34.758018 2606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.133:35591, user_credentials={real_user=slave}} blocked reactor thread for 74312.5us
I20240501 07:59:34.945693 2387 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:35.033629 2782 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:35.038170 2520 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:35.038276 2912 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:35.070986 3043 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
W20240501 07:59:35.130400 2606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.99.129:42933, user_credentials={real_user=slave}} blocked reactor thread for 81264.7us
I20240501 07:59:35.135040 3850 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:35.321130 2649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 07:59:35.771011 3043 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 07:59:35.778311 2649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 07:59:35.778568 2782 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:35.783113 2520 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:35.785132 2387 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:35.801792 2912 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:35.804702 3850 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
9c220cab46b743088853d0027749db0d | 127.0.99.190:35233 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:36633 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
1537338fbeea47ee81f7a210cd6df2f0 | 127.0.99.134:44027 | HEALTHY | <none> | 2 | 0
3bd6898f790b48d593253180b79c5e40 | 127.0.99.132:40829 | HEALTHY | <none> | 1 | 0
5810fd56ce9244438edcd3c0ea23e72c | 127.0.99.130:32773 | HEALTHY | <none> | 1 | 0
5dd2fd1701284a6b886ab8d14d1bd8df | 127.0.99.131:44519 | HEALTHY | <none> | 4 | 0
85377a976c05408fbb384237a0edb691 | 127.0.99.129:42933 | HEALTHY | <none> | 3 | 0
e2c508eb3dac4193a1eeba799e5d54f2 | 127.0.99.133:35591 | HEALTHY | <none> | 5 | 0
e48105f8f6ab476196dd66c4b12808d8 | 127.0.99.135:37389 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 7
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 7 server(s) checked
local_ip_for_outbound_sockets | 127.0.99.129 | experimental | 127.0.99.129:42933
local_ip_for_outbound_sockets | 127.0.99.130 | experimental | 127.0.99.130:32773
local_ip_for_outbound_sockets | 127.0.99.131 | experimental | 127.0.99.131:44519
local_ip_for_outbound_sockets | 127.0.99.132 | experimental | 127.0.99.132:40829
local_ip_for_outbound_sockets | 127.0.99.133 | experimental | 127.0.99.133:35591
local_ip_for_outbound_sockets | 127.0.99.134 | experimental | 127.0.99.134:44027
local_ip_for_outbound_sockets | 127.0.99.135 | experimental | 127.0.99.135:37389
never_fsync | true | unsafe,advanced | all 7 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 7 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 7 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 7 server(s) checked
server_dump_info_format | pb | hidden | all 7 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.0.99.129:42933
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.0.99.130:32773
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.0.99.131:44519
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.0.99.132:40829
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.0.99.133:35591
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.0.99.134:44027
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.0.99.135:37389
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:36633 | all 7 server(s) checked
time_source | builtin | all 7 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 8 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 5 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 11
First Quartile | 11
Median | 11
Third Quartile | 12
Maximum | 12
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 7
Tables | 6
Tablets | 16
Replicas | 80
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240501 07:59:35.843142 398 log_verifier.cc:126] Checking tablet 1ae5c568fb854f2a9e4122fc21314024
I20240501 07:59:35.945463 398 log_verifier.cc:177] Verified matching terms for 179 ops in tablet 1ae5c568fb854f2a9e4122fc21314024
I20240501 07:59:35.945636 398 log_verifier.cc:126] Checking tablet 2a62d5460d7844ebb51f4b33f915bd74
I20240501 07:59:36.056090 398 log_verifier.cc:177] Verified matching terms for 203 ops in tablet 2a62d5460d7844ebb51f4b33f915bd74
I20240501 07:59:36.056253 398 log_verifier.cc:126] Checking tablet 2fcfbbe5535f45ebad6733432d0f284e
I20240501 07:59:36.196957 398 log_verifier.cc:177] Verified matching terms for 273 ops in tablet 2fcfbbe5535f45ebad6733432d0f284e
I20240501 07:59:36.197391 398 log_verifier.cc:126] Checking tablet 382e9b7922f849f980b80ded29e99075
I20240501 07:59:36.304436 398 log_verifier.cc:177] Verified matching terms for 181 ops in tablet 382e9b7922f849f980b80ded29e99075
I20240501 07:59:36.304653 398 log_verifier.cc:126] Checking tablet 3ade42f3f20e4deda01e89ee6ee4141f
I20240501 07:59:36.408533 398 log_verifier.cc:177] Verified matching terms for 179 ops in tablet 3ade42f3f20e4deda01e89ee6ee4141f
I20240501 07:59:36.408689 398 log_verifier.cc:126] Checking tablet 490523aeb9e44f4ba8c2fc42fb73a95e
I20240501 07:59:36.522737 398 log_verifier.cc:177] Verified matching terms for 179 ops in tablet 490523aeb9e44f4ba8c2fc42fb73a95e
I20240501 07:59:36.522886 398 log_verifier.cc:126] Checking tablet 5414588c25d048ed84ec9aae6f300ddf
I20240501 07:59:36.632421 398 log_verifier.cc:177] Verified matching terms for 190 ops in tablet 5414588c25d048ed84ec9aae6f300ddf
I20240501 07:59:36.632571 398 log_verifier.cc:126] Checking tablet 6949f70a1ae44576b42d388231f0b50f
I20240501 07:59:36.769224 398 log_verifier.cc:177] Verified matching terms for 259 ops in tablet 6949f70a1ae44576b42d388231f0b50f
I20240501 07:59:36.769399 398 log_verifier.cc:126] Checking tablet 902e7d877089496689096005187548fe
I20240501 07:59:36.893107 398 log_verifier.cc:177] Verified matching terms for 250 ops in tablet 902e7d877089496689096005187548fe
I20240501 07:59:36.893282 398 log_verifier.cc:126] Checking tablet 917c88788fbf4089a5df0f271cc478f7
I20240501 07:59:36.995754 398 log_verifier.cc:177] Verified matching terms for 185 ops in tablet 917c88788fbf4089a5df0f271cc478f7
I20240501 07:59:36.995920 398 log_verifier.cc:126] Checking tablet 97a598d6cbb84578ab82e3807b98f36f
I20240501 07:59:37.089977 398 log_verifier.cc:177] Verified matching terms for 150 ops in tablet 97a598d6cbb84578ab82e3807b98f36f
I20240501 07:59:37.090116 398 log_verifier.cc:126] Checking tablet a42ea1bbca294f3e9298d895eee087c7
I20240501 07:59:37.209723 398 log_verifier.cc:177] Verified matching terms for 223 ops in tablet a42ea1bbca294f3e9298d895eee087c7
I20240501 07:59:37.209890 398 log_verifier.cc:126] Checking tablet cb4496116793436f849ea95bcc6fc6ba
I20240501 07:59:37.251703 398 log_verifier.cc:177] Verified matching terms for 4 ops in tablet cb4496116793436f849ea95bcc6fc6ba
I20240501 07:59:37.251775 398 log_verifier.cc:126] Checking tablet ceea369d9dcb4a8c94ec88adde948ef6
I20240501 07:59:37.358693 398 log_verifier.cc:177] Verified matching terms for 179 ops in tablet ceea369d9dcb4a8c94ec88adde948ef6
I20240501 07:59:37.358840 398 log_verifier.cc:126] Checking tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa
I20240501 07:59:37.494767 398 log_verifier.cc:177] Verified matching terms for 271 ops in tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa
I20240501 07:59:37.494946 398 log_verifier.cc:126] Checking tablet f64014c3e7474e21b65082390fc1e6a6
I20240501 07:59:37.618378 398 log_verifier.cc:177] Verified matching terms for 246 ops in tablet f64014c3e7474e21b65082390fc1e6a6
I20240501 07:59:37.620342 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.136:0
--local_ip_for_outbound_sockets=127.0.99.136
--webserver_interface=127.0.99.136
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--builtin_ntp_servers=127.0.99.148:36633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 07:59:37.731231 4920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:37.731423 4920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:37.731468 4920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:37.734779 4920 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:37.734841 4920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:37.734916 4920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.136
I20240501 07:59:37.738870 4920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:36633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.136:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.0.99.136
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35233
--never_fsync=true
--heap_profile_path=/tmp/kudu.4920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.136
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:37.739718 4920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:37.740800 4920 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:37.749079 4928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:37.749598 4920 server_base.cc:1034] running on GCE node
W20240501 07:59:37.750100 4926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:37.750519 4925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:37.750842 4920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:37.751327 4920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:37.752465 4920 hybrid_clock.cc:648] HybridClock initialized: now 1714550377752439 us; error 38 us; skew 500 ppm
I20240501 07:59:37.759598 4920 webserver.cc:460] Webserver started at http://127.0.99.136:35625/ using document root <none> and password file <none>
I20240501 07:59:37.760203 4920 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:37.760263 4920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:37.760426 4920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:37.762009 4920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "72d590cc1b1c4fbc86fc4b340d0c484a"
format_stamp: "Formatted at 2024-05-01 07:59:37 on dist-test-slave-f81v"
I20240501 07:59:37.762450 4920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "72d590cc1b1c4fbc86fc4b340d0c484a"
format_stamp: "Formatted at 2024-05-01 07:59:37 on dist-test-slave-f81v"
I20240501 07:59:37.765882 4920 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 07:59:37.768098 4934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:37.769207 4920 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:37.769297 4920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "72d590cc1b1c4fbc86fc4b340d0c484a"
format_stamp: "Formatted at 2024-05-01 07:59:37 on dist-test-slave-f81v"
I20240501 07:59:37.769387 4920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:37.785315 4920 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:37.785939 4920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:37.786093 4920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:37.786659 4920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:37.787540 4920 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:37.787587 4920 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:37.787629 4920 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:37.787657 4920 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:37.795692 4920 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.136:45877
I20240501 07:59:37.795791 5047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.136:45877 every 8 connection(s)
I20240501 07:59:37.796669 4920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 07:59:37.805308 5048 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35233
I20240501 07:59:37.805516 5048 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:37.805969 5048 heartbeater.cc:511] Master 127.0.99.190:35233 requested a full tablet report, sending...
I20240501 07:59:37.806303 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 4920
I20240501 07:59:37.806416 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240501 07:59:37.806902 2264 ts_manager.cc:194] Registered new tserver with Master: 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136:45877)
I20240501 07:59:37.807554 2264 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.136:42031
I20240501 07:59:37.995492 2782 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:37.995831 3043 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 07:59:37.997006 2649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 07:59:38.002264 2912 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:38.010872 2520 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:38.013540 4982 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:38.018754 3850 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:38.026760 2387 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:38.120911 3043 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 07:59:38.126334 2649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 07:59:38.126745 2782 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:38.129387 2520 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:38.132812 2387 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:38.149606 3850 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:38.150972 4982 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:38.151518 2912 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:38.194979 2929 consensus_queue.cc:237] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 246, Committed index: 246, Last appended: 1.246, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 247 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.197266 2404 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 246. Preceding OpId from leader: term: 1 index: 247. (index mismatch)
I20240501 07:59:38.197302 3867 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 246. Preceding OpId from leader: term: 1 index: 247. (index mismatch)
I20240501 07:59:38.197690 5144 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 247, Last known committed idx: 246, Time since last communication: 0.000s
I20240501 07:59:38.197788 3062 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 246. Preceding OpId from leader: term: 1 index: 247. (index mismatch)
I20240501 07:59:38.198176 5145 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 247, Last known committed idx: 246, Time since last communication: 0.000s
I20240501 07:59:38.198383 2796 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 246. Preceding OpId from leader: term: 1 index: 247. (index mismatch)
I20240501 07:59:38.198706 5145 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 247, Last known committed idx: 246, Time since last communication: 0.000s
I20240501 07:59:38.199239 5145 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 247, Last known committed idx: 246, Time since last communication: 0.000s
I20240501 07:59:38.201246 5143 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.247: config changed from index 69 to 247, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 247 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.201443 3062 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.247: config changed from index 69 to 247, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 247 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.202534 2404 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.247: config changed from index 69 to 247, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 247 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.203218 3867 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.247: config changed from index 69 to 247, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 247 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.203967 2266 catalog_manager.cc:5484] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 reported cstate change: config changed from index 69 to 247, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 247 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.204974 2796 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.247: config changed from index 69 to 247, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 247 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
W20240501 07:59:38.205638 2869 consensus_peers.cc:485] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 -> Peer 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136:45877): Couldn't send request to peer 72d590cc1b1c4fbc86fc4b340d0c484a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f64014c3e7474e21b65082390fc1e6a6. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:38.216099 3062 consensus_queue.cc:237] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 179, Committed index: 179, Last appended: 1.179, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.217737 2540 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 180. (index mismatch)
I20240501 07:59:38.218122 2670 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 180. (index mismatch)
I20240501 07:59:38.218221 2404 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 180. (index mismatch)
I20240501 07:59:38.218299 2929 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 180. (index mismatch)
I20240501 07:59:38.218653 5164 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 180, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 07:59:38.218822 4868 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 180, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 07:59:38.218974 5162 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 180, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 07:59:38.219116 5163 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 180, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 07:59:38.220937 5164 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 LEADER]: Committing config change with OpId 1.180: config changed from index -1 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.221318 2404 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.180: config changed from index -1 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.223063 2670 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.180: config changed from index -1 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
W20240501 07:59:38.223486 3000 consensus_peers.cc:485] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 -> Peer 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136:45877): Couldn't send request to peer 72d590cc1b1c4fbc86fc4b340d0c484a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ceea369d9dcb4a8c94ec88adde948ef6. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:38.223431 2261 catalog_manager.cc:5484] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index -1 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "1537338fbeea47ee81f7a210cd6df2f0" committed_config { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.224862 2929 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.180: config changed from index -1 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.226500 2540 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.180: config changed from index -1 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.233201 2404 consensus_queue.cc:237] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 179, Committed index: 179, Last appended: 1.179, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.234930 3062 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 180. (index mismatch)
I20240501 07:59:38.234963 2929 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 180. (index mismatch)
I20240501 07:59:38.235155 3867 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 180. (index mismatch)
I20240501 07:59:38.235247 2670 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 180. (index mismatch)
I20240501 07:59:38.235574 5178 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 180, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 07:59:38.235790 4918 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 180, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 07:59:38.235962 5177 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 180, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 07:59:38.236297 5178 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 180, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 07:59:38.238065 5176 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Committing config change with OpId 1.180: config changed from index 42 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
W20240501 07:59:38.238657 2345 consensus_peers.cc:485] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 -> Peer 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136:45877): Couldn't send request to peer 72d590cc1b1c4fbc86fc4b340d0c484a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1ae5c568fb854f2a9e4122fc21314024. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:38.239806 2670 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.180: config changed from index 42 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.240067 2929 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.180: config changed from index 42 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.241911 2260 catalog_manager.cc:5484] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index 42 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.241987 3867 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.180: config changed from index 42 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.244614 3062 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.180: config changed from index 42 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.250041 2929 consensus_queue.cc:237] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.251663 3867 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:38.251765 2404 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:38.252079 2540 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:38.252221 5190 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:38.252424 5144 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 07:59:38.252573 2869 consensus_peers.cc:485] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 -> Peer 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136:45877): Couldn't send request to peer 72d590cc1b1c4fbc86fc4b340d0c484a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cb4496116793436f849ea95bcc6fc6ba. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:38.252591 3062 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:38.252736 5145 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:38.253031 5190 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:38.254752 4195 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.255106 2404 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.255136 3062 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.256352 3867 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.256940 2540 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.258001 2258 catalog_manager.cc:5484] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 reported cstate change: config changed from index 4 to 5, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.267426 2929 consensus_queue.cc:237] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 223, Committed index: 223, Last appended: 1.223, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.268815 3867 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 223. Preceding OpId from leader: term: 1 index: 224. (index mismatch)
I20240501 07:59:38.268815 2540 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 223. Preceding OpId from leader: term: 1 index: 224. (index mismatch)
W20240501 07:59:38.269161 2869 consensus_peers.cc:485] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 -> Peer 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136:45877): Couldn't send request to peer 72d590cc1b1c4fbc86fc4b340d0c484a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a42ea1bbca294f3e9298d895eee087c7. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:38.269351 5143 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 224, Last known committed idx: 223, Time since last communication: 0.000s
I20240501 07:59:38.269443 2670 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 223. Preceding OpId from leader: term: 1 index: 224. (index mismatch)
I20240501 07:59:38.269487 2796 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 223. Preceding OpId from leader: term: 1 index: 224. (index mismatch)
I20240501 07:59:38.269840 5144 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 224, Last known committed idx: 223, Time since last communication: 0.000s
I20240501 07:59:38.270004 5190 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 224, Last known committed idx: 223, Time since last communication: 0.000s
I20240501 07:59:38.270277 5144 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 224, Last known committed idx: 223, Time since last communication: 0.000s
I20240501 07:59:38.271834 5143 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.224: config changed from index 79 to 224, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.272363 2796 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.224: config changed from index 79 to 224, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.273846 2670 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.224: config changed from index 79 to 224, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.274592 2261 catalog_manager.cc:5484] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: config changed from index 79 to 224, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:38.275272 3867 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.224: config changed from index 79 to 224, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.275858 2540 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.224: config changed from index 79 to 224, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:38.626102 5256 ts_tablet_manager.cc:902] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Initiating tablet copy from peer e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591)
I20240501 07:59:38.626823 5256 tablet_copy_client.cc:308] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:35591
I20240501 07:59:38.630725 2953 tablet_copy_service.cc:140] P e2c508eb3dac4193a1eeba799e5d54f2: Received BeginTabletCopySession request for tablet f64014c3e7474e21b65082390fc1e6a6 from peer 72d590cc1b1c4fbc86fc4b340d0c484a ({username='slave'} at 127.0.99.136:47503)
I20240501 07:59:38.630846 2953 tablet_copy_service.cc:161] P e2c508eb3dac4193a1eeba799e5d54f2: Beginning new tablet copy session on tablet f64014c3e7474e21b65082390fc1e6a6 from peer 72d590cc1b1c4fbc86fc4b340d0c484a at {username='slave'} at 127.0.99.136:47503: session id = 72d590cc1b1c4fbc86fc4b340d0c484a-f64014c3e7474e21b65082390fc1e6a6
I20240501 07:59:38.631588 2953 tablet_copy_source_session.cc:205] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:38.634516 5256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64014c3e7474e21b65082390fc1e6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:38.639179 5256 tablet_copy_client.cc:783] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:38.639355 5256 tablet_copy_client.cc:647] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:38.641585 5256 tablet_copy_client.cc:523] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:38.643138 5256 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap starting.
I20240501 07:59:38.661015 5256 log.cc:824] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:38.674373 5263 ts_tablet_manager.cc:902] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a: Initiating tablet copy from peer 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933)
I20240501 07:59:38.674870 5263 tablet_copy_client.cc:308] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:42933
I20240501 07:59:38.678297 2429 tablet_copy_service.cc:140] P 85377a976c05408fbb384237a0edb691: Received BeginTabletCopySession request for tablet 1ae5c568fb854f2a9e4122fc21314024 from peer 72d590cc1b1c4fbc86fc4b340d0c484a ({username='slave'} at 127.0.99.136:40473)
I20240501 07:59:38.678403 2429 tablet_copy_service.cc:161] P 85377a976c05408fbb384237a0edb691: Beginning new tablet copy session on tablet 1ae5c568fb854f2a9e4122fc21314024 from peer 72d590cc1b1c4fbc86fc4b340d0c484a at {username='slave'} at 127.0.99.136:40473: session id = 72d590cc1b1c4fbc86fc4b340d0c484a-1ae5c568fb854f2a9e4122fc21314024
I20240501 07:59:38.679415 2429 tablet_copy_source_session.cc:205] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:38.680572 5263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae5c568fb854f2a9e4122fc21314024. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:38.682758 5263 tablet_copy_client.cc:783] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:38.682952 5263 tablet_copy_client.cc:647] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:38.684258 5263 tablet_copy_client.cc:523] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:38.685318 5263 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap starting.
I20240501 07:59:38.708117 5269 ts_tablet_manager.cc:902] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a: Initiating tablet copy from peer e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591)
I20240501 07:59:38.708724 5269 tablet_copy_client.cc:308] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:35591
I20240501 07:59:38.709143 2953 tablet_copy_service.cc:140] P e2c508eb3dac4193a1eeba799e5d54f2: Received BeginTabletCopySession request for tablet cb4496116793436f849ea95bcc6fc6ba from peer 72d590cc1b1c4fbc86fc4b340d0c484a ({username='slave'} at 127.0.99.136:47503)
I20240501 07:59:38.709265 2953 tablet_copy_service.cc:161] P e2c508eb3dac4193a1eeba799e5d54f2: Beginning new tablet copy session on tablet cb4496116793436f849ea95bcc6fc6ba from peer 72d590cc1b1c4fbc86fc4b340d0c484a at {username='slave'} at 127.0.99.136:47503: session id = 72d590cc1b1c4fbc86fc4b340d0c484a-cb4496116793436f849ea95bcc6fc6ba
I20240501 07:59:38.709932 2953 tablet_copy_source_session.cc:205] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:38.710872 5269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4496116793436f849ea95bcc6fc6ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:38.712870 5269 tablet_copy_client.cc:783] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:38.713063 5269 tablet_copy_client.cc:647] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:38.713965 5269 tablet_copy_client.cc:523] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:38.714956 5269 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap starting.
I20240501 07:59:38.718482 5269 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:38.718736 5269 tablet_bootstrap.cc:492] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap complete.
I20240501 07:59:38.719007 5269 ts_tablet_manager.cc:1367] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 07:59:38.721437 5269 raft_consensus.cc:348] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.721663 5269 raft_consensus.cc:729] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 72d590cc1b1c4fbc86fc4b340d0c484a, State: Initialized, Role: LEARNER
I20240501 07:59:38.721987 5269 consensus_queue.cc:260] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.722633 5048 heartbeater.cc:503] Master 127.0.99.190:35233 was elected leader, sending a full tablet report...
I20240501 07:59:38.722856 5269 ts_tablet_manager.cc:1398] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20240501 07:59:38.723421 2953 tablet_copy_service.cc:342] P e2c508eb3dac4193a1eeba799e5d54f2: Request end of tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-cb4496116793436f849ea95bcc6fc6ba received from {username='slave'} at 127.0.99.136:47503
I20240501 07:59:38.723493 2953 tablet_copy_service.cc:433] P e2c508eb3dac4193a1eeba799e5d54f2: ending tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-cb4496116793436f849ea95bcc6fc6ba on tablet cb4496116793436f849ea95bcc6fc6ba with peer 72d590cc1b1c4fbc86fc4b340d0c484a
I20240501 07:59:38.728343 5269 ts_tablet_manager.cc:902] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Initiating tablet copy from peer 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027)
I20240501 07:59:38.728835 5269 tablet_copy_client.cc:308] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:44027
I20240501 07:59:38.733384 3084 tablet_copy_service.cc:140] P 1537338fbeea47ee81f7a210cd6df2f0: Received BeginTabletCopySession request for tablet ceea369d9dcb4a8c94ec88adde948ef6 from peer 72d590cc1b1c4fbc86fc4b340d0c484a ({username='slave'} at 127.0.99.136:49283)
I20240501 07:59:38.733486 3084 tablet_copy_service.cc:161] P 1537338fbeea47ee81f7a210cd6df2f0: Beginning new tablet copy session on tablet ceea369d9dcb4a8c94ec88adde948ef6 from peer 72d590cc1b1c4fbc86fc4b340d0c484a at {username='slave'} at 127.0.99.136:49283: session id = 72d590cc1b1c4fbc86fc4b340d0c484a-ceea369d9dcb4a8c94ec88adde948ef6
I20240501 07:59:38.734227 3084 tablet_copy_source_session.cc:205] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:38.735294 5269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceea369d9dcb4a8c94ec88adde948ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:38.737753 5269 tablet_copy_client.cc:783] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:38.737896 5269 tablet_copy_client.cc:647] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:38.739951 5269 tablet_copy_client.cc:523] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:38.741020 5269 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap starting.
I20240501 07:59:38.818567 5256 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap replayed 1/1 log segments. Stats: ops{read=247 overwritten=0 applied=247 ignored=0} inserts{seen=916 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:38.818921 5256 tablet_bootstrap.cc:492] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap complete.
I20240501 07:59:38.819175 5256 ts_tablet_manager.cc:1367] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent bootstrapping tablet: real 0.176s user 0.166s sys 0.008s
I20240501 07:59:38.819690 5256 raft_consensus.cc:348] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 247 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.819799 5256 raft_consensus.cc:729] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 72d590cc1b1c4fbc86fc4b340d0c484a, State: Initialized, Role: LEARNER
I20240501 07:59:38.819892 5256 consensus_queue.cc:260] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 247, Last appended: 1.247, Last appended by leader: 247, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 247 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.820191 5256 ts_tablet_manager.cc:1398] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 07:59:38.820664 2953 tablet_copy_service.cc:342] P e2c508eb3dac4193a1eeba799e5d54f2: Request end of tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-f64014c3e7474e21b65082390fc1e6a6 received from {username='slave'} at 127.0.99.136:47503
I20240501 07:59:38.820756 2953 tablet_copy_service.cc:433] P e2c508eb3dac4193a1eeba799e5d54f2: ending tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-f64014c3e7474e21b65082390fc1e6a6 on tablet f64014c3e7474e21b65082390fc1e6a6 with peer 72d590cc1b1c4fbc86fc4b340d0c484a
I20240501 07:59:38.848820 5263 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap replayed 1/1 log segments. Stats: ops{read=180 overwritten=0 applied=180 ignored=0} inserts{seen=862 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:38.849169 5263 tablet_bootstrap.cc:492] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap complete.
I20240501 07:59:38.849395 5263 ts_tablet_manager.cc:1367] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent bootstrapping tablet: real 0.164s user 0.136s sys 0.009s
I20240501 07:59:38.849916 5263 raft_consensus.cc:348] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.850078 5263 raft_consensus.cc:729] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 72d590cc1b1c4fbc86fc4b340d0c484a, State: Initialized, Role: LEARNER
I20240501 07:59:38.850211 5263 consensus_queue.cc:260] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 180, Last appended: 1.180, Last appended by leader: 180, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.850538 5263 ts_tablet_manager.cc:1398] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:38.851083 2429 tablet_copy_service.cc:342] P 85377a976c05408fbb384237a0edb691: Request end of tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-1ae5c568fb854f2a9e4122fc21314024 received from {username='slave'} at 127.0.99.136:40473
I20240501 07:59:38.851171 2429 tablet_copy_service.cc:433] P 85377a976c05408fbb384237a0edb691: ending tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-1ae5c568fb854f2a9e4122fc21314024 on tablet 1ae5c568fb854f2a9e4122fc21314024 with peer 72d590cc1b1c4fbc86fc4b340d0c484a
I20240501 07:59:38.892050 5263 ts_tablet_manager.cc:902] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a: Initiating tablet copy from peer e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591)
I20240501 07:59:38.892513 5263 tablet_copy_client.cc:308] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:35591
I20240501 07:59:38.892974 2953 tablet_copy_service.cc:140] P e2c508eb3dac4193a1eeba799e5d54f2: Received BeginTabletCopySession request for tablet a42ea1bbca294f3e9298d895eee087c7 from peer 72d590cc1b1c4fbc86fc4b340d0c484a ({username='slave'} at 127.0.99.136:47503)
I20240501 07:59:38.893083 2953 tablet_copy_service.cc:161] P e2c508eb3dac4193a1eeba799e5d54f2: Beginning new tablet copy session on tablet a42ea1bbca294f3e9298d895eee087c7 from peer 72d590cc1b1c4fbc86fc4b340d0c484a at {username='slave'} at 127.0.99.136:47503: session id = 72d590cc1b1c4fbc86fc4b340d0c484a-a42ea1bbca294f3e9298d895eee087c7
I20240501 07:59:38.893791 2953 tablet_copy_source_session.cc:205] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:38.894585 5263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42ea1bbca294f3e9298d895eee087c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:38.894790 5269 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap replayed 1/1 log segments. Stats: ops{read=180 overwritten=0 applied=180 ignored=0} inserts{seen=852 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:38.895022 5269 tablet_bootstrap.cc:492] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap complete.
I20240501 07:59:38.895195 5269 ts_tablet_manager.cc:1367] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent bootstrapping tablet: real 0.154s user 0.123s sys 0.009s
I20240501 07:59:38.895614 5269 raft_consensus.cc:348] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.895715 5269 raft_consensus.cc:729] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 72d590cc1b1c4fbc86fc4b340d0c484a, State: Initialized, Role: LEARNER
I20240501 07:59:38.895831 5269 consensus_queue.cc:260] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 180, Last appended: 1.180, Last appended by leader: 180, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:38.896122 5269 ts_tablet_manager.cc:1398] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 07:59:38.896598 3084 tablet_copy_service.cc:342] P 1537338fbeea47ee81f7a210cd6df2f0: Request end of tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-ceea369d9dcb4a8c94ec88adde948ef6 received from {username='slave'} at 127.0.99.136:49283
I20240501 07:59:38.896672 3084 tablet_copy_service.cc:433] P 1537338fbeea47ee81f7a210cd6df2f0: ending tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-ceea369d9dcb4a8c94ec88adde948ef6 on tablet ceea369d9dcb4a8c94ec88adde948ef6 with peer 72d590cc1b1c4fbc86fc4b340d0c484a
I20240501 07:59:38.896678 5263 tablet_copy_client.cc:783] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:38.896826 5263 tablet_copy_client.cc:647] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:38.898252 5263 tablet_copy_client.cc:523] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:38.899513 5263 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap starting.
I20240501 07:59:39.035152 5263 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap replayed 1/1 log segments. Stats: ops{read=224 overwritten=0 applied=224 ignored=0} inserts{seen=852 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:39.035413 5263 tablet_bootstrap.cc:492] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap complete.
I20240501 07:59:39.035598 5263 ts_tablet_manager.cc:1367] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent bootstrapping tablet: real 0.136s user 0.129s sys 0.003s
I20240501 07:59:39.036023 5263 raft_consensus.cc:348] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:39.036144 5263 raft_consensus.cc:729] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 72d590cc1b1c4fbc86fc4b340d0c484a, State: Initialized, Role: LEARNER
I20240501 07:59:39.036235 5263 consensus_queue.cc:260] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 224, Last appended: 1.224, Last appended by leader: 224, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:39.036466 5263 ts_tablet_manager.cc:1398] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 07:59:39.036949 2953 tablet_copy_service.cc:342] P e2c508eb3dac4193a1eeba799e5d54f2: Request end of tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-a42ea1bbca294f3e9298d895eee087c7 received from {username='slave'} at 127.0.99.136:47503
I20240501 07:59:39.037022 2953 tablet_copy_service.cc:433] P e2c508eb3dac4193a1eeba799e5d54f2: ending tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-a42ea1bbca294f3e9298d895eee087c7 on tablet a42ea1bbca294f3e9298d895eee087c7 with peer 72d590cc1b1c4fbc86fc4b340d0c484a
I20240501 07:59:39.058148 5002 raft_consensus.cc:1203] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.179->[1.180-1.180] Dedup: 1.180->[]
I20240501 07:59:39.133090 5002 raft_consensus.cc:1203] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.246->[1.247-1.247] Dedup: 1.247->[]
I20240501 07:59:39.133488 5001 raft_consensus.cc:1203] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.179->[1.180-1.180] Dedup: 1.180->[]
I20240501 07:59:39.176697 5001 raft_consensus.cc:1203] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 07:59:39.458233 5001 raft_consensus.cc:1203] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.223->[1.224-1.224] Dedup: 1.224->[]
I20240501 07:59:39.468339 5176 raft_consensus.cc:1050] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691: attempting to promote NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a to VOTER
I20240501 07:59:39.468808 5176 consensus_queue.cc:237] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 180, Committed index: 180, Last appended: 1.180, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:39.470522 3062 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:39.470697 3867 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:39.471007 5001 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:39.471074 2666 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:39.471503 5376 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:39.471560 2929 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:39.471776 5378 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:39.471971 5377 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:39.472299 5177 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:39.472519 5176 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:39.474867 5378 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.475303 3867 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.476999 2929 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.477912 2666 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.478250 2258 catalog_manager.cc:5484] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 reported cstate change: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.478124 5001 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.482295 3062 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.487330 2404 consensus_queue.cc:237] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 181, Committed index: 181, Last appended: 1.181, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:39.488679 3867 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240501 07:59:39.488916 2666 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240501 07:59:39.488983 3062 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240501 07:59:39.489382 5378 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 07:59:39.489959 5176 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 07:59:39.490284 5176 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 07:59:39.490998 5001 raft_consensus.cc:1261] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240501 07:59:39.491397 5176 consensus_queue.cc:1035] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 07:59:39.491860 5176 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.493333 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1ae5c568fb854f2a9e4122fc21314024 with cas_config_opid_index 181: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:39.493333 5001 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.493935 2666 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.494675 3867 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.495473 2265 catalog_manager.cc:5484] T 1ae5c568fb854f2a9e4122fc21314024 P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index 181 to 182, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:39.497279 3062 raft_consensus.cc:2932] T 1ae5c568fb854f2a9e4122fc21314024 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.499132 2912 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ae5c568fb854f2a9e4122fc21314024 with delete_type TABLET_DATA_TOMBSTONED (TS e2c508eb3dac4193a1eeba799e5d54f2 not found in new config with opid_index 182) from {username='slave'} at 127.0.0.1:48460
I20240501 07:59:39.499526 5404 tablet_replica.cc:330] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2: stopping tablet replica
I20240501 07:59:39.499660 5404 raft_consensus.cc:2229] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:39.499955 5404 raft_consensus.cc:2258] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:39.501590 5404 ts_tablet_manager.cc:1875] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:39.503618 5404 ts_tablet_manager.cc:1888] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.181
I20240501 07:59:39.503679 5404 log.cc:1196] T 1ae5c568fb854f2a9e4122fc21314024 P e2c508eb3dac4193a1eeba799e5d54f2: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/1ae5c568fb854f2a9e4122fc21314024
I20240501 07:59:39.504146 2253 catalog_manager.cc:4830] TS e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591): tablet 1ae5c568fb854f2a9e4122fc21314024 (table rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]) successfully deleted
I20240501 07:59:39.587724 5339 raft_consensus.cc:1050] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2: attempting to promote NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a to VOTER
I20240501 07:59:39.588227 5339 consensus_queue.cc:237] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:39.589881 2404 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:39.589977 5001 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:39.590015 2540 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:39.590354 5406 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:39.590510 3867 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:39.590566 5407 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:39.590837 5336 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:39.590920 3062 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:39.591183 5190 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:39.592088 5190 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:39.593362 5406 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.593780 3867 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.593837 2404 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.595446 5001 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.595957 2540 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.596449 2266 catalog_manager.cc:5484] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index 5 to 6, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.596776 3062 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.601357 2929 consensus_queue.cc:237] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:39.603018 5001 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 07:59:39.603018 3867 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 07:59:39.603416 5406 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 07:59:39.603596 5190 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 07:59:39.603664 2404 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 07:59:39.603999 5190 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 07:59:39.604869 2540 raft_consensus.cc:1261] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 07:59:39.605271 5190 consensus_queue.cc:1035] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 07:59:39.605587 5190 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.605979 2404 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.606005 5001 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.607124 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet cb4496116793436f849ea95bcc6fc6ba with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:39.607910 3867 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.608342 2264 catalog_manager.cc:5484] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: config changed from index 6 to 7, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:39.609896 2540 raft_consensus.cc:2932] T cb4496116793436f849ea95bcc6fc6ba P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.612187 3043 tablet_service.cc:1514] Processing DeleteTablet for tablet cb4496116793436f849ea95bcc6fc6ba with delete_type TABLET_DATA_TOMBSTONED (TS 1537338fbeea47ee81f7a210cd6df2f0 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:60186
I20240501 07:59:39.612584 5413 tablet_replica.cc:330] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0: stopping tablet replica
I20240501 07:59:39.612694 5413 raft_consensus.cc:2229] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:39.612901 5413 raft_consensus.cc:2258] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:39.614122 5413 ts_tablet_manager.cc:1875] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:39.615876 5413 ts_tablet_manager.cc:1888] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 07:59:39.615971 5413 log.cc:1196] T cb4496116793436f849ea95bcc6fc6ba P 1537338fbeea47ee81f7a210cd6df2f0: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/cb4496116793436f849ea95bcc6fc6ba
I20240501 07:59:39.616397 2254 catalog_manager.cc:4830] TS 1537338fbeea47ee81f7a210cd6df2f0 (127.0.99.134:44027): tablet cb4496116793436f849ea95bcc6fc6ba (table TestTable [id=8e1fca9f5d8e40cd8da5d523ca3d4e09]) successfully deleted
I20240501 07:59:39.676517 5338 raft_consensus.cc:1050] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0: attempting to promote NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a to VOTER
I20240501 07:59:39.676963 5338 consensus_queue.cc:237] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 180, Committed index: 180, Last appended: 1.180, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:39.678633 5001 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:39.678731 2404 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:39.678880 2540 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:39.678877 2666 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:39.679288 2931 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:39.679360 5417 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:39.679616 5416 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:39.679773 5418 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:39.679966 5414 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:39.680173 5419 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:39.682880 5417 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 LEADER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.683404 5001 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.683467 2540 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.685412 2931 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.685828 2265 catalog_manager.cc:5484] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a reported cstate change: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1537338fbeea47ee81f7a210cd6df2f0" committed_config { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.685984 2404 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.686542 2666 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.690410 3062 consensus_queue.cc:237] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 181, Committed index: 181, Last appended: 1.181, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:39.691809 5001 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240501 07:59:39.691990 2540 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240501 07:59:39.692490 5417 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 07:59:39.692988 5414 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 07:59:39.693058 2404 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240501 07:59:39.693550 5417 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 07:59:39.694139 2931 raft_consensus.cc:1261] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 1537338fbeea47ee81f7a210cd6df2f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240501 07:59:39.694489 5417 consensus_queue.cc:1035] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 07:59:39.694877 5417 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 LEADER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.695245 5001 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.695768 2404 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.696494 2931 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.697772 2261 catalog_manager.cc:5484] T ceea369d9dcb4a8c94ec88adde948ef6 P 72d590cc1b1c4fbc86fc4b340d0c484a reported cstate change: config changed from index 181 to 182, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "1537338fbeea47ee81f7a210cd6df2f0" committed_config { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.698653 2254 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ceea369d9dcb4a8c94ec88adde948ef6 with cas_config_opid_index 181: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:39.699288 2540 raft_consensus.cc:2932] T ceea369d9dcb4a8c94ec88adde948ef6 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.701004 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet ceea369d9dcb4a8c94ec88adde948ef6 with delete_type TABLET_DATA_TOMBSTONED (TS 5dd2fd1701284a6b886ab8d14d1bd8df not found in new config with opid_index 182) from {username='slave'} at 127.0.0.1:44588
I20240501 07:59:39.701530 5425 tablet_replica.cc:330] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df: stopping tablet replica
I20240501 07:59:39.701663 5425 raft_consensus.cc:2229] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:39.701822 5425 raft_consensus.cc:2258] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:39.704541 5425 ts_tablet_manager.cc:1875] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:39.706804 5425 ts_tablet_manager.cc:1888] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.181
I20240501 07:59:39.706887 5425 log.cc:1196] T ceea369d9dcb4a8c94ec88adde948ef6 P 5dd2fd1701284a6b886ab8d14d1bd8df: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/ceea369d9dcb4a8c94ec88adde948ef6
I20240501 07:59:39.707556 2252 catalog_manager.cc:4830] TS 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519): tablet ceea369d9dcb4a8c94ec88adde948ef6 (table rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]) successfully deleted
I20240501 07:59:39.747211 2666 consensus_queue.cc:237] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 190, Committed index: 190, Last appended: 1.190, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 191 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:39.748979 2540 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 190. Preceding OpId from leader: term: 1 index: 191. (index mismatch)
I20240501 07:59:39.749523 5459 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 191, Last known committed idx: 190, Time since last communication: 0.000s
I20240501 07:59:39.749563 2931 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 190. Preceding OpId from leader: term: 1 index: 191. (index mismatch)
I20240501 07:59:39.749888 2796 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 190. Preceding OpId from leader: term: 1 index: 191. (index mismatch)
I20240501 07:59:39.749943 5458 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 191, Last known committed idx: 190, Time since last communication: 0.000s
I20240501 07:59:39.750233 3867 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 190. Preceding OpId from leader: term: 1 index: 191. (index mismatch)
I20240501 07:59:39.750756 5459 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 191, Last known committed idx: 190, Time since last communication: 0.000s
I20240501 07:59:39.750946 4919 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 191, Last known committed idx: 190, Time since last communication: 0.000s
I20240501 07:59:39.752877 5458 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.191: config changed from index 62 to 191, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 191 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.753188 2540 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.191: config changed from index 62 to 191, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 191 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.754931 2796 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.191: config changed from index 62 to 191, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 191 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.755220 2931 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.191: config changed from index 62 to 191, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 191 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.755518 2264 catalog_manager.cc:5484] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index 62 to 191, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 191 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:39.757766 2607 consensus_peers.cc:485] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df -> Peer 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136:45877): Couldn't send request to peer 72d590cc1b1c4fbc86fc4b340d0c484a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5414588c25d048ed84ec9aae6f300ddf. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:39.758229 3867 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.191: config changed from index 62 to 191, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 191 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.758816 5336 raft_consensus.cc:1050] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2: attempting to promote NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a to VOTER
I20240501 07:59:39.759263 5336 consensus_queue.cc:237] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 247, Committed index: 247, Last appended: 1.247, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 248 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:39.760973 5001 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 247. Preceding OpId from leader: term: 1 index: 248. (index mismatch)
I20240501 07:59:39.761014 3867 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 247. Preceding OpId from leader: term: 1 index: 248. (index mismatch)
I20240501 07:59:39.761448 5190 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 248, Last known committed idx: 247, Time since last communication: 0.000s
I20240501 07:59:39.761682 2404 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 247. Preceding OpId from leader: term: 1 index: 248. (index mismatch)
I20240501 07:59:39.762058 3062 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 247. Preceding OpId from leader: term: 1 index: 248. (index mismatch)
I20240501 07:59:39.762609 2796 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 247. Preceding OpId from leader: term: 1 index: 248. (index mismatch)
I20240501 07:59:39.762866 5339 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 248, Last known committed idx: 247, Time since last communication: 0.000s
I20240501 07:59:39.763005 5406 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 248, Last known committed idx: 247, Time since last communication: 0.000s
I20240501 07:59:39.763317 5406 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 248, Last known committed idx: 247, Time since last communication: 0.000s
I20240501 07:59:39.763648 5190 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 248, Last known committed idx: 247, Time since last communication: 0.000s
I20240501 07:59:39.766388 5406 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.248: config changed from index 247 to 248, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 248 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.767737 2796 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.248: config changed from index 247 to 248, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 248 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.768458 5001 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.248: config changed from index 247 to 248, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 248 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.768522 3867 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.248: config changed from index 247 to 248, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 248 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.769979 2404 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.248: config changed from index 247 to 248, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 248 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.770613 2261 catalog_manager.cc:5484] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a reported cstate change: config changed from index 247 to 248, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 248 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.771903 3062 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.248: config changed from index 247 to 248, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 248 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.774647 2931 consensus_queue.cc:237] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 248, Committed index: 248, Last appended: 1.248, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 249 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:39.775812 2666 consensus_queue.cc:237] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 179, Committed index: 179, Last appended: 1.179, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:39.776773 3867 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 248. Preceding OpId from leader: term: 1 index: 249. (index mismatch)
I20240501 07:59:39.777163 5339 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 249, Last known committed idx: 248, Time since last communication: 0.000s
I20240501 07:59:39.777761 5001 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 248. Preceding OpId from leader: term: 1 index: 249. (index mismatch)
W20240501 07:59:39.777863 2607 consensus_peers.cc:485] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df -> Peer 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136:45877): Couldn't send request to peer 72d590cc1b1c4fbc86fc4b340d0c484a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 490523aeb9e44f4ba8c2fc42fb73a95e. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:39.777912 2540 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 180. (index mismatch)
I20240501 07:59:39.778142 2408 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 180. (index mismatch)
I20240501 07:59:39.778282 5339 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 249, Last known committed idx: 248, Time since last communication: 0.000s
I20240501 07:59:39.778367 5458 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 180, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 07:59:39.778468 2931 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 180. (index mismatch)
I20240501 07:59:39.778569 5460 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 180, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 07:59:39.779093 2796 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 180. (index mismatch)
I20240501 07:59:39.779120 3062 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 248. Preceding OpId from leader: term: 1 index: 249. (index mismatch)
I20240501 07:59:39.779191 5458 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 180, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 07:59:39.779596 2404 raft_consensus.cc:1261] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 248. Preceding OpId from leader: term: 1 index: 249. (index mismatch)
I20240501 07:59:39.779774 5460 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 180, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 07:59:39.780512 5339 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 249, Last known committed idx: 248, Time since last communication: 0.000s
I20240501 07:59:39.781019 5339 consensus_queue.cc:1035] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 249, Last known committed idx: 248, Time since last communication: 0.000s
I20240501 07:59:39.782280 5460 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.180: config changed from index 65 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.782680 5339 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.249: config changed from index 248 to 249, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 249 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.784343 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f64014c3e7474e21b65082390fc1e6a6 with cas_config_opid_index 248: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:39.785173 2796 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.180: config changed from index 65 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.786082 2265 catalog_manager.cc:5484] T f64014c3e7474e21b65082390fc1e6a6 P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: config changed from index 248 to 249, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 249 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:39.788028 2260 catalog_manager.cc:5484] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 reported cstate change: config changed from index 65 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.788193 2540 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.180: config changed from index 65 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.790409 3867 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.249: config changed from index 248 to 249, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 249 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.793895 5001 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.249: config changed from index 248 to 249, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 249 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.793888 2931 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.180: config changed from index 65 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.798107 2540 consensus_queue.cc:237] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 273, Committed index: 273, Last appended: 1.273, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:39.800280 2666 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 273. Preceding OpId from leader: term: 1 index: 274. (index mismatch)
I20240501 07:59:39.800401 3867 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 273. Preceding OpId from leader: term: 1 index: 274. (index mismatch)
I20240501 07:59:39.800750 2796 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 273. Preceding OpId from leader: term: 1 index: 274. (index mismatch)
I20240501 07:59:39.800820 4407 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 274, Last known committed idx: 273, Time since last communication: 0.000s
I20240501 07:59:39.801157 4407 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 274, Last known committed idx: 273, Time since last communication: 0.000s
I20240501 07:59:39.801172 2408 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.249: config changed from index 248 to 249, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 249 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.802062 2931 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 273. Preceding OpId from leader: term: 1 index: 274. (index mismatch)
I20240501 07:59:39.802631 4407 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 274, Last known committed idx: 273, Time since last communication: 0.000s
I20240501 07:59:39.802946 4407 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 274, Last known committed idx: 273, Time since last communication: 0.000s
I20240501 07:59:39.803100 3062 raft_consensus.cc:2932] T f64014c3e7474e21b65082390fc1e6a6 P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.249: config changed from index 248 to 249, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 249 OBSOLETE_local: false peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:39.805550 4407 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 LEADER]: Committing config change with OpId 1.274: config changed from index 134 to 274, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
W20240501 07:59:39.808159 2476 consensus_peers.cc:485] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c -> Peer 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136:45877): Couldn't send request to peer 72d590cc1b1c4fbc86fc4b340d0c484a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2fcfbbe5535f45ebad6733432d0f284e. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:39.808786 2782 tablet_service.cc:1514] Processing DeleteTablet for tablet f64014c3e7474e21b65082390fc1e6a6 with delete_type TABLET_DATA_TOMBSTONED (TS 3bd6898f790b48d593253180b79c5e40 not found in new config with opid_index 249) from {username='slave'} at 127.0.0.1:51830
I20240501 07:59:39.809582 2666 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.274: config changed from index 134 to 274, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.810387 3867 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.274: config changed from index 134 to 274, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.812098 5500 tablet_replica.cc:330] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40: stopping tablet replica
I20240501 07:59:39.812189 2260 catalog_manager.cc:5484] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index 134 to 274, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "5810fd56ce9244438edcd3c0ea23e72c" committed_config { opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.812284 5500 raft_consensus.cc:2229] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:39.812505 5500 raft_consensus.cc:2258] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:39.814500 5500 ts_tablet_manager.cc:1875] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:39.816123 2796 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.274: config changed from index 134 to 274, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.817459 5500 ts_tablet_manager.cc:1888] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.248
I20240501 07:59:39.817584 5500 log.cc:1196] T f64014c3e7474e21b65082390fc1e6a6 P 3bd6898f790b48d593253180b79c5e40: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/f64014c3e7474e21b65082390fc1e6a6
I20240501 07:59:39.818187 2253 catalog_manager.cc:4830] TS 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829): tablet f64014c3e7474e21b65082390fc1e6a6 (table rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]) successfully deleted
I20240501 07:59:39.819592 2931 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.274: config changed from index 134 to 274, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.819743 2404 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.180: config changed from index 65 to 180, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.854104 2931 consensus_queue.cc:237] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 150, Committed index: 150, Last appended: 1.150, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 151 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:39.855710 2796 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 150. Preceding OpId from leader: term: 1 index: 151. (index mismatch)
W20240501 07:59:39.855985 2869 consensus_peers.cc:485] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 -> Peer 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136:45877): Couldn't send request to peer 72d590cc1b1c4fbc86fc4b340d0c484a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 97a598d6cbb84578ab82e3807b98f36f. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:39.856175 2404 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 150. Preceding OpId from leader: term: 1 index: 151. (index mismatch)
I20240501 07:59:39.856175 2540 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 150. Preceding OpId from leader: term: 1 index: 151. (index mismatch)
I20240501 07:59:39.856582 5336 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 151, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 07:59:39.856729 2666 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 150. Preceding OpId from leader: term: 1 index: 151. (index mismatch)
I20240501 07:59:39.856735 5406 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 151, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 07:59:39.856912 5190 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 151, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 07:59:39.857079 5143 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 151, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 07:59:39.859122 5143 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.151: config changed from index -1 to 151, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 151 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.860584 2796 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.151: config changed from index -1 to 151, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 151 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.861264 2666 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.151: config changed from index -1 to 151, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 151 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.861615 2404 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.151: config changed from index -1 to 151, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 151 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:39.863322 2265 catalog_manager.cc:5484] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: config changed from index -1 to 151, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 151 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:39.865729 2540 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.151: config changed from index -1 to 151, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 151 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:40.063052 5407 raft_consensus.cc:1050] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2: attempting to promote NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a to VOTER
I20240501 07:59:40.063498 5407 consensus_queue.cc:237] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 224, Committed index: 224, Last appended: 1.224, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:40.065040 2540 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 224. Preceding OpId from leader: term: 1 index: 225. (index mismatch)
I20240501 07:59:40.065089 5001 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 224. Preceding OpId from leader: term: 1 index: 225. (index mismatch)
I20240501 07:59:40.065119 2666 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 224. Preceding OpId from leader: term: 1 index: 225. (index mismatch)
I20240501 07:59:40.065469 5190 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 225, Last known committed idx: 224, Time since last communication: 0.000s
I20240501 07:59:40.065781 3867 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 224. Preceding OpId from leader: term: 1 index: 225. (index mismatch)
I20240501 07:59:40.065902 2796 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 224. Preceding OpId from leader: term: 1 index: 225. (index mismatch)
I20240501 07:59:40.065887 5143 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 225, Last known committed idx: 224, Time since last communication: 0.000s
I20240501 07:59:40.066131 5406 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 225, Last known committed idx: 224, Time since last communication: 0.000s
I20240501 07:59:40.066771 5406 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 225, Last known committed idx: 224, Time since last communication: 0.000s
I20240501 07:59:40.066943 5143 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 225, Last known committed idx: 224, Time since last communication: 0.000s
I20240501 07:59:40.068321 5143 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.225: config changed from index 224 to 225, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.069165 2666 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.225: config changed from index 224 to 225, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.069339 2796 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.225: config changed from index 224 to 225, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.070363 2540 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.225: config changed from index 224 to 225, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.071851 5001 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.225: config changed from index 224 to 225, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.072028 2261 catalog_manager.cc:5484] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index 224 to 225, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.073303 3867 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.225: config changed from index 224 to 225, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.081250 2931 consensus_queue.cc:237] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 225, Committed index: 225, Last appended: 1.225, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 226 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:40.082552 3867 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 225. Preceding OpId from leader: term: 1 index: 226. (index mismatch)
I20240501 07:59:40.082680 2666 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 225. Preceding OpId from leader: term: 1 index: 226. (index mismatch)
I20240501 07:59:40.082682 5001 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 225. Preceding OpId from leader: term: 1 index: 226. (index mismatch)
I20240501 07:59:40.083017 5143 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 226, Last known committed idx: 225, Time since last communication: 0.000s
I20240501 07:59:40.083359 2796 raft_consensus.cc:1261] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 225. Preceding OpId from leader: term: 1 index: 226. (index mismatch)
I20240501 07:59:40.083518 5336 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 226, Last known committed idx: 225, Time since last communication: 0.000s
I20240501 07:59:40.083679 5406 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 226, Last known committed idx: 225, Time since last communication: 0.000s
I20240501 07:59:40.084028 5406 consensus_queue.cc:1035] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 226, Last known committed idx: 225, Time since last communication: 0.000s
I20240501 07:59:40.085625 5406 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.226: config changed from index 225 to 226, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 226 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.086519 2666 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.226: config changed from index 225 to 226, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 226 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.086519 5001 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.226: config changed from index 225 to 226, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 226 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.088455 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a42ea1bbca294f3e9298d895eee087c7 with cas_config_opid_index 225: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:40.089553 2264 catalog_manager.cc:5484] T a42ea1bbca294f3e9298d895eee087c7 P 72d590cc1b1c4fbc86fc4b340d0c484a reported cstate change: config changed from index 225 to 226, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 226 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.090611 3867 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.226: config changed from index 225 to 226, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 226 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.092039 2796 raft_consensus.cc:2932] T a42ea1bbca294f3e9298d895eee087c7 P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.226: config changed from index 225 to 226, VOTER 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130) evicted. New config: { opid_index: 226 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:40.092345 2520 tablet_service.cc:1514] Processing DeleteTablet for tablet a42ea1bbca294f3e9298d895eee087c7 with delete_type TABLET_DATA_TOMBSTONED (TS 5810fd56ce9244438edcd3c0ea23e72c not found in new config with opid_index 226) from {username='slave'} at 127.0.0.1:56996
I20240501 07:59:40.094528 5567 tablet_replica.cc:330] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c: stopping tablet replica
I20240501 07:59:40.094880 5567 raft_consensus.cc:2229] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:40.095156 5567 raft_consensus.cc:2258] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:40.097363 5567 ts_tablet_manager.cc:1875] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:40.099694 5567 ts_tablet_manager.cc:1888] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.225
I20240501 07:59:40.099777 5567 log.cc:1196] T a42ea1bbca294f3e9298d895eee087c7 P 5810fd56ce9244438edcd3c0ea23e72c: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/a42ea1bbca294f3e9298d895eee087c7
I20240501 07:59:40.100328 2251 catalog_manager.cc:4830] TS 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773): tablet a42ea1bbca294f3e9298d895eee087c7 (table rebalance_test_table_3 [id=d87957cf347545a38a6298fa134ae83b]) successfully deleted
I20240501 07:59:40.133697 2404 consensus_queue.cc:237] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 271, Committed index: 271, Last appended: 1.271, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 272 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:40.135531 2796 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 271. Preceding OpId from leader: term: 1 index: 272. (index mismatch)
I20240501 07:59:40.135530 2540 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 271. Preceding OpId from leader: term: 1 index: 272. (index mismatch)
I20240501 07:59:40.135561 3062 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 271. Preceding OpId from leader: term: 1 index: 272. (index mismatch)
W20240501 07:59:40.135915 2345 consensus_peers.cc:485] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 -> Peer 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136:45877): Couldn't send request to peer 72d590cc1b1c4fbc86fc4b340d0c484a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cf2bcc9afe2f4c839e80fadb7fcf4dfa. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:40.135901 2931 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 271. Preceding OpId from leader: term: 1 index: 272. (index mismatch)
I20240501 07:59:40.136150 5600 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 272, Last known committed idx: 271, Time since last communication: 0.000s
I20240501 07:59:40.136394 5599 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 272, Last known committed idx: 271, Time since last communication: 0.000s
I20240501 07:59:40.136586 5602 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 272, Last known committed idx: 271, Time since last communication: 0.000s
I20240501 07:59:40.136744 5601 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 272, Last known committed idx: 271, Time since last communication: 0.000s
I20240501 07:59:40.138675 5599 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Committing config change with OpId 1.272: config changed from index -1 to 272, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 272 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:40.138980 3062 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.272: config changed from index -1 to 272, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 272 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:40.139267 2796 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.272: config changed from index -1 to 272, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 272 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:40.140172 2931 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.272: config changed from index -1 to 272, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 272 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:40.141049 2540 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.272: config changed from index -1 to 272, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New config: { opid_index: 272 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:40.141924 2264 catalog_manager.cc:5484] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 reported cstate change: config changed from index -1 to 272, NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: 272 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } } }
I20240501 07:59:40.270546 5632 ts_tablet_manager.cc:902] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a: Initiating tablet copy from peer 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519)
I20240501 07:59:40.271162 5632 tablet_copy_client.cc:308] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:44519
I20240501 07:59:40.274788 2691 tablet_copy_service.cc:140] P 5dd2fd1701284a6b886ab8d14d1bd8df: Received BeginTabletCopySession request for tablet 5414588c25d048ed84ec9aae6f300ddf from peer 72d590cc1b1c4fbc86fc4b340d0c484a ({username='slave'} at 127.0.99.136:57863)
I20240501 07:59:40.274894 2691 tablet_copy_service.cc:161] P 5dd2fd1701284a6b886ab8d14d1bd8df: Beginning new tablet copy session on tablet 5414588c25d048ed84ec9aae6f300ddf from peer 72d590cc1b1c4fbc86fc4b340d0c484a at {username='slave'} at 127.0.99.136:57863: session id = 72d590cc1b1c4fbc86fc4b340d0c484a-5414588c25d048ed84ec9aae6f300ddf
I20240501 07:59:40.275596 2691 tablet_copy_source_session.cc:205] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:40.276392 5632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5414588c25d048ed84ec9aae6f300ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:40.280583 5632 tablet_copy_client.cc:783] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:40.280714 5632 tablet_copy_client.cc:647] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:40.282241 5632 tablet_copy_client.cc:523] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:40.283286 5632 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap starting.
I20240501 07:59:40.325163 5639 ts_tablet_manager.cc:902] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a: Initiating tablet copy from peer e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591)
I20240501 07:59:40.325683 5639 tablet_copy_client.cc:308] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:35591
I20240501 07:59:40.326229 2953 tablet_copy_service.cc:140] P e2c508eb3dac4193a1eeba799e5d54f2: Received BeginTabletCopySession request for tablet 97a598d6cbb84578ab82e3807b98f36f from peer 72d590cc1b1c4fbc86fc4b340d0c484a ({username='slave'} at 127.0.99.136:47503)
I20240501 07:59:40.326355 2953 tablet_copy_service.cc:161] P e2c508eb3dac4193a1eeba799e5d54f2: Beginning new tablet copy session on tablet 97a598d6cbb84578ab82e3807b98f36f from peer 72d590cc1b1c4fbc86fc4b340d0c484a at {username='slave'} at 127.0.99.136:47503: session id = 72d590cc1b1c4fbc86fc4b340d0c484a-97a598d6cbb84578ab82e3807b98f36f
I20240501 07:59:40.327404 2953 tablet_copy_source_session.cc:205] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:40.328248 5639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97a598d6cbb84578ab82e3807b98f36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:40.330030 5639 tablet_copy_client.cc:783] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:40.330157 5639 tablet_copy_client.cc:647] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:40.331492 5639 tablet_copy_client.cc:523] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:40.332672 5639 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap starting.
I20240501 07:59:40.392496 5655 ts_tablet_manager.cc:902] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a: Initiating tablet copy from peer 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519)
I20240501 07:59:40.393031 5655 tablet_copy_client.cc:308] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:44519
I20240501 07:59:40.393759 2691 tablet_copy_service.cc:140] P 5dd2fd1701284a6b886ab8d14d1bd8df: Received BeginTabletCopySession request for tablet 490523aeb9e44f4ba8c2fc42fb73a95e from peer 72d590cc1b1c4fbc86fc4b340d0c484a ({username='slave'} at 127.0.99.136:57863)
I20240501 07:59:40.393874 2691 tablet_copy_service.cc:161] P 5dd2fd1701284a6b886ab8d14d1bd8df: Beginning new tablet copy session on tablet 490523aeb9e44f4ba8c2fc42fb73a95e from peer 72d590cc1b1c4fbc86fc4b340d0c484a at {username='slave'} at 127.0.99.136:57863: session id = 72d590cc1b1c4fbc86fc4b340d0c484a-490523aeb9e44f4ba8c2fc42fb73a95e
I20240501 07:59:40.394830 2691 tablet_copy_source_session.cc:205] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:40.400343 5655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 490523aeb9e44f4ba8c2fc42fb73a95e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:40.403301 5655 tablet_copy_client.cc:783] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:40.403448 5655 tablet_copy_client.cc:647] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:40.405653 5655 tablet_copy_client.cc:523] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:40.407558 5655 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap starting.
I20240501 07:59:40.409462 5632 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap replayed 1/1 log segments. Stats: ops{read=191 overwritten=0 applied=191 ignored=0} inserts{seen=873 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:40.409699 5632 tablet_bootstrap.cc:492] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap complete.
I20240501 07:59:40.409857 5632 ts_tablet_manager.cc:1367] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent bootstrapping tablet: real 0.127s user 0.110s sys 0.015s
I20240501 07:59:40.410329 5632 raft_consensus.cc:348] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 191 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:40.410457 5632 raft_consensus.cc:729] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 72d590cc1b1c4fbc86fc4b340d0c484a, State: Initialized, Role: LEARNER
I20240501 07:59:40.410552 5632 consensus_queue.cc:260] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 191, Last appended: 1.191, Last appended by leader: 191, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 191 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:40.416673 5674 ts_tablet_manager.cc:902] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a: Initiating tablet copy from peer 5810fd56ce9244438edcd3c0ea23e72c (127.0.99.130:32773)
I20240501 07:59:40.417315 5674 tablet_copy_client.cc:308] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:32773
I20240501 07:59:40.423442 2560 tablet_copy_service.cc:140] P 5810fd56ce9244438edcd3c0ea23e72c: Received BeginTabletCopySession request for tablet 2fcfbbe5535f45ebad6733432d0f284e from peer 72d590cc1b1c4fbc86fc4b340d0c484a ({username='slave'} at 127.0.99.136:54851)
I20240501 07:59:40.423575 2560 tablet_copy_service.cc:161] P 5810fd56ce9244438edcd3c0ea23e72c: Beginning new tablet copy session on tablet 2fcfbbe5535f45ebad6733432d0f284e from peer 72d590cc1b1c4fbc86fc4b340d0c484a at {username='slave'} at 127.0.99.136:54851: session id = 72d590cc1b1c4fbc86fc4b340d0c484a-2fcfbbe5535f45ebad6733432d0f284e
I20240501 07:59:40.424757 2560 tablet_copy_source_session.cc:205] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:40.425766 5674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fcfbbe5535f45ebad6733432d0f284e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:40.428148 5632 ts_tablet_manager.cc:1398] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20240501 07:59:40.430359 2691 tablet_copy_service.cc:342] P 5dd2fd1701284a6b886ab8d14d1bd8df: Request end of tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-5414588c25d048ed84ec9aae6f300ddf received from {username='slave'} at 127.0.99.136:57863
I20240501 07:59:40.430436 2691 tablet_copy_service.cc:433] P 5dd2fd1701284a6b886ab8d14d1bd8df: ending tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-5414588c25d048ed84ec9aae6f300ddf on tablet 5414588c25d048ed84ec9aae6f300ddf with peer 72d590cc1b1c4fbc86fc4b340d0c484a
I20240501 07:59:40.431000 5674 tablet_copy_client.cc:783] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:40.431245 5674 tablet_copy_client.cc:647] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:40.434111 5674 tablet_copy_client.cc:523] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:40.435817 5674 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap starting.
I20240501 07:59:40.452980 5639 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap replayed 1/1 log segments. Stats: ops{read=151 overwritten=0 applied=151 ignored=0} inserts{seen=836 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:40.453284 5639 tablet_bootstrap.cc:492] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap complete.
I20240501 07:59:40.453485 5639 ts_tablet_manager.cc:1367] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent bootstrapping tablet: real 0.121s user 0.106s sys 0.004s
I20240501 07:59:40.454057 5639 raft_consensus.cc:348] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 151 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:40.454207 5639 raft_consensus.cc:729] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 72d590cc1b1c4fbc86fc4b340d0c484a, State: Initialized, Role: LEARNER
I20240501 07:59:40.454319 5639 consensus_queue.cc:260] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 151, Last appended: 1.151, Last appended by leader: 151, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 151 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:40.454636 5639 ts_tablet_manager.cc:1398] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:40.455380 2953 tablet_copy_service.cc:342] P e2c508eb3dac4193a1eeba799e5d54f2: Request end of tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-97a598d6cbb84578ab82e3807b98f36f received from {username='slave'} at 127.0.99.136:47503
I20240501 07:59:40.455468 2953 tablet_copy_service.cc:433] P e2c508eb3dac4193a1eeba799e5d54f2: ending tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-97a598d6cbb84578ab82e3807b98f36f on tablet 97a598d6cbb84578ab82e3807b98f36f with peer 72d590cc1b1c4fbc86fc4b340d0c484a
I20240501 07:59:40.537585 5655 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap replayed 1/1 log segments. Stats: ops{read=180 overwritten=0 applied=180 ignored=0} inserts{seen=849 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:40.537895 5655 tablet_bootstrap.cc:492] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap complete.
I20240501 07:59:40.538080 5655 ts_tablet_manager.cc:1367] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent bootstrapping tablet: real 0.131s user 0.097s sys 0.012s
I20240501 07:59:40.538497 5655 raft_consensus.cc:348] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:40.538606 5655 raft_consensus.cc:729] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 72d590cc1b1c4fbc86fc4b340d0c484a, State: Initialized, Role: LEARNER
I20240501 07:59:40.538710 5655 consensus_queue.cc:260] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 180, Last appended: 1.180, Last appended by leader: 180, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 180 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:40.538983 5655 ts_tablet_manager.cc:1398] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:40.539600 2691 tablet_copy_service.cc:342] P 5dd2fd1701284a6b886ab8d14d1bd8df: Request end of tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-490523aeb9e44f4ba8c2fc42fb73a95e received from {username='slave'} at 127.0.99.136:57863
I20240501 07:59:40.539697 2691 tablet_copy_service.cc:433] P 5dd2fd1701284a6b886ab8d14d1bd8df: ending tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-490523aeb9e44f4ba8c2fc42fb73a95e on tablet 490523aeb9e44f4ba8c2fc42fb73a95e with peer 72d590cc1b1c4fbc86fc4b340d0c484a
I20240501 07:59:40.602488 5655 ts_tablet_manager.cc:902] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a: Initiating tablet copy from peer 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933)
I20240501 07:59:40.602922 5655 tablet_copy_client.cc:308] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:42933
I20240501 07:59:40.603453 2429 tablet_copy_service.cc:140] P 85377a976c05408fbb384237a0edb691: Received BeginTabletCopySession request for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa from peer 72d590cc1b1c4fbc86fc4b340d0c484a ({username='slave'} at 127.0.99.136:40473)
I20240501 07:59:40.603583 2429 tablet_copy_service.cc:161] P 85377a976c05408fbb384237a0edb691: Beginning new tablet copy session on tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa from peer 72d590cc1b1c4fbc86fc4b340d0c484a at {username='slave'} at 127.0.99.136:40473: session id = 72d590cc1b1c4fbc86fc4b340d0c484a-cf2bcc9afe2f4c839e80fadb7fcf4dfa
I20240501 07:59:40.604454 2429 tablet_copy_source_session.cc:205] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:40.605212 5655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:40.606936 5655 tablet_copy_client.cc:783] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:40.607084 5655 tablet_copy_client.cc:647] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:40.608575 5655 tablet_copy_client.cc:523] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:40.609611 5655 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap starting.
I20240501 07:59:40.613852 5674 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap replayed 1/1 log segments. Stats: ops{read=274 overwritten=0 applied=274 ignored=0} inserts{seen=956 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:40.614101 5674 tablet_bootstrap.cc:492] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap complete.
I20240501 07:59:40.614282 5674 ts_tablet_manager.cc:1367] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent bootstrapping tablet: real 0.179s user 0.165s sys 0.008s
I20240501 07:59:40.614670 5674 raft_consensus.cc:348] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:40.614779 5674 raft_consensus.cc:729] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 72d590cc1b1c4fbc86fc4b340d0c484a, State: Initialized, Role: LEARNER
I20240501 07:59:40.614871 5674 consensus_queue.cc:260] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 274, Last appended: 1.274, Last appended by leader: 274, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:40.615128 5674 ts_tablet_manager.cc:1398] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:40.615624 2560 tablet_copy_service.cc:342] P 5810fd56ce9244438edcd3c0ea23e72c: Request end of tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-2fcfbbe5535f45ebad6733432d0f284e received from {username='slave'} at 127.0.99.136:54851
I20240501 07:59:40.615711 2560 tablet_copy_service.cc:433] P 5810fd56ce9244438edcd3c0ea23e72c: ending tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-2fcfbbe5535f45ebad6733432d0f284e on tablet 2fcfbbe5535f45ebad6733432d0f284e with peer 72d590cc1b1c4fbc86fc4b340d0c484a
I20240501 07:59:40.777007 5001 raft_consensus.cc:1203] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.190->[1.191-1.191] Dedup: 1.191->[]
I20240501 07:59:40.778590 5655 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap replayed 1/1 log segments. Stats: ops{read=272 overwritten=0 applied=272 ignored=0} inserts{seen=944 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:40.778872 5655 tablet_bootstrap.cc:492] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a: Bootstrap complete.
I20240501 07:59:40.779085 5655 ts_tablet_manager.cc:1367] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent bootstrapping tablet: real 0.170s user 0.153s sys 0.012s
I20240501 07:59:40.779558 5655 raft_consensus.cc:348] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 272 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:40.779670 5655 raft_consensus.cc:729] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 72d590cc1b1c4fbc86fc4b340d0c484a, State: Initialized, Role: LEARNER
I20240501 07:59:40.779767 5655 consensus_queue.cc:260] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 272, Last appended: 1.272, Last appended by leader: 272, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 272 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: true } }
I20240501 07:59:40.780084 5655 ts_tablet_manager.cc:1398] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 07:59:40.780617 2429 tablet_copy_service.cc:342] P 85377a976c05408fbb384237a0edb691: Request end of tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-cf2bcc9afe2f4c839e80fadb7fcf4dfa received from {username='slave'} at 127.0.99.136:40473
I20240501 07:59:40.780691 2429 tablet_copy_service.cc:433] P 85377a976c05408fbb384237a0edb691: ending tablet copy session 72d590cc1b1c4fbc86fc4b340d0c484a-cf2bcc9afe2f4c839e80fadb7fcf4dfa on tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa with peer 72d590cc1b1c4fbc86fc4b340d0c484a
I20240501 07:59:40.803625 5001 raft_consensus.cc:1203] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.179->[1.180-1.180] Dedup: 1.180->[]
I20240501 07:59:40.851963 5001 raft_consensus.cc:1203] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.150->[1.151-1.151] Dedup: 1.151->[]
I20240501 07:59:41.033922 5001 raft_consensus.cc:1203] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.273->[1.274-1.274] Dedup: 1.274->[]
I20240501 07:59:41.049880 5001 raft_consensus.cc:1203] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.271->[1.272-1.272] Dedup: 1.272->[]
I20240501 07:59:41.324606 5336 raft_consensus.cc:1050] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2: attempting to promote NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a to VOTER
I20240501 07:59:41.325006 5336 consensus_queue.cc:237] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 151, Committed index: 151, Last appended: 1.151, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:41.326646 2796 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 07:59:41.326661 2540 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 07:59:41.326817 2666 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 07:59:41.326810 2404 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 07:59:41.327037 5001 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 07:59:41.327184 5770 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 151, Time since last communication: 0.000s
I20240501 07:59:41.327358 5190 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 151, Time since last communication: 0.000s
I20240501 07:59:41.327505 5336 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 151, Time since last communication: 0.000s
I20240501 07:59:41.327662 5714 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 151, Time since last communication: 0.000s
I20240501 07:59:41.327864 5771 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 151, Time since last communication: 0.000s
I20240501 07:59:41.330703 5190 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.152: config changed from index 151 to 152, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.331445 2540 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 151 to 152, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.332212 2666 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 151 to 152, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.332545 2404 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 151 to 152, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.335099 2265 catalog_manager.cc:5484] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index 151 to 152, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.336221 2796 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 151 to 152, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.336905 5001 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 151 to 152, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } attrs { replace: true } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.342516 2931 consensus_queue.cc:237] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 152, Committed index: 152, Last appended: 1.152, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:41.344218 2540 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240501 07:59:41.344230 2796 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240501 07:59:41.344781 5001 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240501 07:59:41.344807 2404 raft_consensus.cc:1261] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer e2c508eb3dac4193a1eeba799e5d54f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240501 07:59:41.344834 5770 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240501 07:59:41.345052 5771 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240501 07:59:41.345225 5770 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240501 07:59:41.345429 5714 consensus_queue.cc:1035] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240501 07:59:41.347797 5771 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 LEADER]: Committing config change with OpId 1.153: config changed from index 152 to 153, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.348114 5001 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.349746 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 97a598d6cbb84578ab82e3807b98f36f with cas_config_opid_index 152: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:41.350221 2404 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.350338 2540 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.350555 2796 raft_consensus.cc:2932] T 97a598d6cbb84578ab82e3807b98f36f P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.350823 2264 catalog_manager.cc:5484] T 97a598d6cbb84578ab82e3807b98f36f P 72d590cc1b1c4fbc86fc4b340d0c484a reported cstate change: config changed from index 152 to 153, VOTER 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" committed_config { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.355083 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet 97a598d6cbb84578ab82e3807b98f36f with delete_type TABLET_DATA_TOMBSTONED (TS 5dd2fd1701284a6b886ab8d14d1bd8df not found in new config with opid_index 153) from {username='slave'} at 127.0.0.1:44588
I20240501 07:59:41.355430 5791 tablet_replica.cc:330] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df: stopping tablet replica
I20240501 07:59:41.355799 5791 raft_consensus.cc:2229] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:41.356079 5791 raft_consensus.cc:2258] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:41.357867 5791 ts_tablet_manager.cc:1875] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:41.360033 5791 ts_tablet_manager.cc:1888] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.152
I20240501 07:59:41.360112 5791 log.cc:1196] T 97a598d6cbb84578ab82e3807b98f36f P 5dd2fd1701284a6b886ab8d14d1bd8df: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/97a598d6cbb84578ab82e3807b98f36f
I20240501 07:59:41.361019 2252 catalog_manager.cc:4830] TS 5dd2fd1701284a6b886ab8d14d1bd8df (127.0.99.131:44519): tablet 97a598d6cbb84578ab82e3807b98f36f (table rebalance_test_table_2 [id=de8b9d2a0ca34b6cbbbbd54c29e15b74]) successfully deleted
I20240501 07:59:41.380667 5812 raft_consensus.cc:1050] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df: attempting to promote NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a to VOTER
I20240501 07:59:41.381553 5812 consensus_queue.cc:237] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 191, Committed index: 191, Last appended: 1.191, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 192 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:41.384052 5001 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 191. Preceding OpId from leader: term: 1 index: 192. (index mismatch)
I20240501 07:59:41.384542 5713 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 192, Last known committed idx: 191, Time since last communication: 0.000s
I20240501 07:59:41.385506 2540 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 191. Preceding OpId from leader: term: 1 index: 192. (index mismatch)
I20240501 07:59:41.385712 3867 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 191. Preceding OpId from leader: term: 1 index: 192. (index mismatch)
I20240501 07:59:41.386312 5819 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 192, Last known committed idx: 191, Time since last communication: 0.000s
I20240501 07:59:41.386648 5819 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 192, Last known committed idx: 191, Time since last communication: 0.000s
I20240501 07:59:41.389091 5819 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.192: config changed from index 191 to 192, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 192 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.389478 5001 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.192: config changed from index 191 to 192, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 192 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.392060 2266 catalog_manager.cc:5484] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a reported cstate change: config changed from index 191 to 192, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 192 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.392706 2931 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 191. Preceding OpId from leader: term: 1 index: 192. (index mismatch)
I20240501 07:59:41.392747 2796 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 191. Preceding OpId from leader: term: 1 index: 192. (index mismatch)
I20240501 07:59:41.393173 5824 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 192, Last known committed idx: 191, Time since last communication: 0.000s
I20240501 07:59:41.393481 2540 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.192: config changed from index 191 to 192, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 192 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.393839 3867 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.192: config changed from index 191 to 192, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 192 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.394178 5713 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 192, Last known committed idx: 191, Time since last communication: 0.000s
I20240501 07:59:41.395218 2796 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.192: config changed from index 191 to 192, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 192 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.395694 2931 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.192: config changed from index 191 to 192, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 192 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false replace: true } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.400113 5713 raft_consensus.cc:1050] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df: attempting to promote NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a to VOTER
I20240501 07:59:41.400594 5713 consensus_queue.cc:237] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 180, Committed index: 180, Last appended: 1.180, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:41.401965 5001 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:41.402141 2540 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:41.402390 2796 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:41.402454 5713 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:41.402559 2929 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:41.402776 2404 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 07:59:41.403313 5713 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:41.403713 5713 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:41.404031 5713 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:41.404191 5836 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 07:59:41.405985 5825 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.406447 2796 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.406672 2404 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.407155 5001 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.408591 2540 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.409299 2266 catalog_manager.cc:5484] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 reported cstate change: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.414340 2670 consensus_queue.cc:237] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 181, Committed index: 181, Last appended: 1.181, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:41.415565 2540 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240501 07:59:41.415714 2931 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 180 to 181, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } attrs { replace: true } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.416054 5001 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240501 07:59:41.416126 5824 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 07:59:41.416504 5824 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 07:59:41.417822 2796 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240501 07:59:41.418267 5825 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 07:59:41.418610 2931 raft_consensus.cc:1261] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 181. Preceding OpId from leader: term: 1 index: 182. (index mismatch)
I20240501 07:59:41.418959 5824 consensus_queue.cc:1035] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 182, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 07:59:41.419286 5824 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.420706 2252 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 490523aeb9e44f4ba8c2fc42fb73a95e with cas_config_opid_index 181: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:41.421854 2796 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.422021 5001 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.422055 2540 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.422808 2931 raft_consensus.cc:2932] T 490523aeb9e44f4ba8c2fc42fb73a95e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.182: config changed from index 181 to 182, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New config: { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.423955 2258 catalog_manager.cc:5484] T 490523aeb9e44f4ba8c2fc42fb73a95e P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index 181 to 182, VOTER 85377a976c05408fbb384237a0edb691 (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 182 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:41.426856 2387 tablet_service.cc:1514] Processing DeleteTablet for tablet 490523aeb9e44f4ba8c2fc42fb73a95e with delete_type TABLET_DATA_TOMBSTONED (TS 85377a976c05408fbb384237a0edb691 not found in new config with opid_index 182) from {username='slave'} at 127.0.0.1:48528
I20240501 07:59:41.430500 5842 tablet_replica.cc:330] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691: stopping tablet replica
I20240501 07:59:41.430910 5842 raft_consensus.cc:2229] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:41.431133 5842 raft_consensus.cc:2258] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:41.433192 5842 ts_tablet_manager.cc:1875] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:41.435384 5842 ts_tablet_manager.cc:1888] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.181
I20240501 07:59:41.435452 5842 log.cc:1196] T 490523aeb9e44f4ba8c2fc42fb73a95e P 85377a976c05408fbb384237a0edb691: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/490523aeb9e44f4ba8c2fc42fb73a95e
I20240501 07:59:41.435976 2253 catalog_manager.cc:4830] TS 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933): tablet 490523aeb9e44f4ba8c2fc42fb73a95e (table rebalance_test_table_1 [id=0d2f2d6395da492e942ab2c1b19de141]) successfully deleted
I20240501 07:59:41.575716 5848 raft_consensus.cc:1050] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691: attempting to promote NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a to VOTER
I20240501 07:59:41.576221 5848 consensus_queue.cc:237] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 272, Committed index: 272, Last appended: 1.272, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 273 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:41.577792 5001 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 272. Preceding OpId from leader: term: 1 index: 273. (index mismatch)
I20240501 07:59:41.577801 2796 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 272. Preceding OpId from leader: term: 1 index: 273. (index mismatch)
I20240501 07:59:41.577927 2540 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 272. Preceding OpId from leader: term: 1 index: 273. (index mismatch)
I20240501 07:59:41.577931 2931 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 272. Preceding OpId from leader: term: 1 index: 273. (index mismatch)
I20240501 07:59:41.578219 5851 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 273, Last known committed idx: 272, Time since last communication: 0.000s
I20240501 07:59:41.578415 3062 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 272. Preceding OpId from leader: term: 1 index: 273. (index mismatch)
I20240501 07:59:41.578460 5853 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 273, Last known committed idx: 272, Time since last communication: 0.000s
I20240501 07:59:41.578706 5851 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 273, Last known committed idx: 272, Time since last communication: 0.000s
I20240501 07:59:41.578941 5601 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 273, Last known committed idx: 272, Time since last communication: 0.000s
I20240501 07:59:41.579186 5850 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 273, Last known committed idx: 272, Time since last communication: 0.000s
I20240501 07:59:41.581595 5601 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Committing config change with OpId 1.273: config changed from index 272 to 273, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 273 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.582106 5001 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.273: config changed from index 272 to 273, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 273 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.582291 2931 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.273: config changed from index 272 to 273, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 273 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.582310 2540 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.273: config changed from index 272 to 273, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 273 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.583904 3062 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.273: config changed from index 272 to 273, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 273 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.585083 2264 catalog_manager.cc:5484] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c reported cstate change: config changed from index 272 to 273, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: 273 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.585618 2796 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.273: config changed from index 272 to 273, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 273 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } attrs { replace: true } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.588707 2404 consensus_queue.cc:237] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 273, Committed index: 273, Last appended: 1.273, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:41.590000 5001 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 273. Preceding OpId from leader: term: 1 index: 274. (index mismatch)
I20240501 07:59:41.590102 2931 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 273. Preceding OpId from leader: term: 1 index: 274. (index mismatch)
I20240501 07:59:41.590354 2540 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 273. Preceding OpId from leader: term: 1 index: 274. (index mismatch)
I20240501 07:59:41.590376 5850 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 274, Last known committed idx: 273, Time since last communication: 0.000s
I20240501 07:59:41.590582 3062 raft_consensus.cc:1261] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Refusing update from remote peer 85377a976c05408fbb384237a0edb691: Log matching property violated. Preceding OpId in replica: term: 1 index: 273. Preceding OpId from leader: term: 1 index: 274. (index mismatch)
I20240501 07:59:41.590582 5601 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 274, Last known committed idx: 273, Time since last communication: 0.000s
I20240501 07:59:41.590821 5850 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 274, Last known committed idx: 273, Time since last communication: 0.000s
I20240501 07:59:41.590960 5601 consensus_queue.cc:1035] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 274, Last known committed idx: 273, Time since last communication: 0.000s
I20240501 07:59:41.593036 5601 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 85377a976c05408fbb384237a0edb691 [term 1 LEADER]: Committing config change with OpId 1.274: config changed from index 273 to 274, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.593432 2931 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.274: config changed from index 273 to 274, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.594799 2253 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa with cas_config_opid_index 273: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:41.595546 5001 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.274: config changed from index 273 to 274, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.595769 2264 catalog_manager.cc:5484] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P e2c508eb3dac4193a1eeba799e5d54f2 reported cstate change: config changed from index 273 to 274, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "85377a976c05408fbb384237a0edb691" committed_config { opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.596045 2540 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.274: config changed from index 273 to 274, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.598557 2782 tablet_service.cc:1514] Processing DeleteTablet for tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa with delete_type TABLET_DATA_TOMBSTONED (TS 3bd6898f790b48d593253180b79c5e40 not found in new config with opid_index 274) from {username='slave'} at 127.0.0.1:51830
I20240501 07:59:41.599014 5859 tablet_replica.cc:330] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40: stopping tablet replica
I20240501 07:59:41.599184 5859 raft_consensus.cc:2229] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:41.599363 5859 raft_consensus.cc:2258] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:41.600976 3062 raft_consensus.cc:2932] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 1537338fbeea47ee81f7a210cd6df2f0 [term 1 FOLLOWER]: Committing config change with OpId 1.274: config changed from index 273 to 274, VOTER 3bd6898f790b48d593253180b79c5e40 (127.0.99.132) evicted. New config: { opid_index: 274 OBSOLETE_local: false peers { permanent_uuid: "1537338fbeea47ee81f7a210cd6df2f0" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 44027 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "85377a976c05408fbb384237a0edb691" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42933 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.602408 5859 ts_tablet_manager.cc:1875] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:41.604399 5859 ts_tablet_manager.cc:1888] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.273
I20240501 07:59:41.604462 5859 log.cc:1196] T cf2bcc9afe2f4c839e80fadb7fcf4dfa P 3bd6898f790b48d593253180b79c5e40: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/cf2bcc9afe2f4c839e80fadb7fcf4dfa
I20240501 07:59:41.604940 2253 catalog_manager.cc:4830] TS 3bd6898f790b48d593253180b79c5e40 (127.0.99.132:40829): tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa (table rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]) successfully deleted
I20240501 07:59:41.641315 5864 raft_consensus.cc:1050] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c: attempting to promote NON_VOTER 72d590cc1b1c4fbc86fc4b340d0c484a to VOTER
I20240501 07:59:41.641759 5864 consensus_queue.cc:237] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 274, Committed index: 274, Last appended: 1.274, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 275 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:41.643538 2796 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 274. Preceding OpId from leader: term: 1 index: 275. (index mismatch)
I20240501 07:59:41.643565 2931 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 274. Preceding OpId from leader: term: 1 index: 275. (index mismatch)
I20240501 07:59:41.643565 2670 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 274. Preceding OpId from leader: term: 1 index: 275. (index mismatch)
I20240501 07:59:41.643697 5001 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 LEARNER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 274. Preceding OpId from leader: term: 1 index: 275. (index mismatch)
I20240501 07:59:41.644071 5864 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 275, Last known committed idx: 274, Time since last communication: 0.000s
I20240501 07:59:41.644155 3867 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 274. Preceding OpId from leader: term: 1 index: 275. (index mismatch)
I20240501 07:59:41.644274 5869 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 275, Last known committed idx: 274, Time since last communication: 0.000s
I20240501 07:59:41.644440 5867 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 275, Last known committed idx: 274, Time since last communication: 0.000s
I20240501 07:59:41.644654 5864 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 275, Last known committed idx: 274, Time since last communication: 0.000s
I20240501 07:59:41.647020 5867 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 275, Last known committed idx: 274, Time since last communication: 0.000s
I20240501 07:59:41.647354 5869 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 LEADER]: Committing config change with OpId 1.275: config changed from index 274 to 275, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 275 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.647780 2796 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.275: config changed from index 274 to 275, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 275 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.650118 2670 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.275: config changed from index 274 to 275, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 275 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.650312 2258 catalog_manager.cc:5484] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c reported cstate change: config changed from index 274 to 275, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5810fd56ce9244438edcd3c0ea23e72c" committed_config { opid_index: 275 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:41.651116 5001 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.275: config changed from index 274 to 275, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 275 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.652420 2931 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.275: config changed from index 274 to 275, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 275 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.654793 3867 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.275: config changed from index 274 to 275, 72d590cc1b1c4fbc86fc4b340d0c484a (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 275 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } attrs { replace: true } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.659556 2540 consensus_queue.cc:237] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 275, Committed index: 275, Last appended: 1.275, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 276 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:41.661104 3867 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 275. Preceding OpId from leader: term: 1 index: 276. (index mismatch)
I20240501 07:59:41.661115 2670 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 275. Preceding OpId from leader: term: 1 index: 276. (index mismatch)
I20240501 07:59:41.661127 5001 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 275. Preceding OpId from leader: term: 1 index: 276. (index mismatch)
I20240501 07:59:41.661146 2796 raft_consensus.cc:1261] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5810fd56ce9244438edcd3c0ea23e72c: Log matching property violated. Preceding OpId in replica: term: 1 index: 275. Preceding OpId from leader: term: 1 index: 276. (index mismatch)
I20240501 07:59:41.661499 5869 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 276, Last known committed idx: 275, Time since last communication: 0.000s
I20240501 07:59:41.661690 5864 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 276, Last known committed idx: 275, Time since last communication: 0.000s
I20240501 07:59:41.662017 5867 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 276, Last known committed idx: 275, Time since last communication: 0.000s
I20240501 07:59:41.662325 5868 consensus_queue.cc:1035] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 276, Last known committed idx: 275, Time since last communication: 0.000s
I20240501 07:59:41.664242 5864 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 5810fd56ce9244438edcd3c0ea23e72c [term 1 LEADER]: Committing config change with OpId 1.276: config changed from index 275 to 276, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 276 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.665022 3867 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Committing config change with OpId 1.276: config changed from index 275 to 276, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 276 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.665074 2796 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.276: config changed from index 275 to 276, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 276 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.665117 2670 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 FOLLOWER]: Committing config change with OpId 1.276: config changed from index 275 to 276, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 276 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.667532 2260 catalog_manager.cc:5484] T 2fcfbbe5535f45ebad6733432d0f284e P 3bd6898f790b48d593253180b79c5e40 reported cstate change: config changed from index 275 to 276, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "5810fd56ce9244438edcd3c0ea23e72c" committed_config { opid_index: 276 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.668120 2251 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2fcfbbe5535f45ebad6733432d0f284e with cas_config_opid_index 275: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:41.670018 5001 raft_consensus.cc:2932] T 2fcfbbe5535f45ebad6733432d0f284e P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.276: config changed from index 275 to 276, VOTER e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133) evicted. New config: { opid_index: 276 OBSOLETE_local: false peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "e48105f8f6ab476196dd66c4b12808d8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 37389 } attrs { promote: false } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.670647 2912 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fcfbbe5535f45ebad6733432d0f284e with delete_type TABLET_DATA_TOMBSTONED (TS e2c508eb3dac4193a1eeba799e5d54f2 not found in new config with opid_index 276) from {username='slave'} at 127.0.0.1:48460
I20240501 07:59:41.672068 5885 tablet_replica.cc:330] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2: stopping tablet replica
I20240501 07:59:41.672349 5885 raft_consensus.cc:2229] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:41.672608 5885 raft_consensus.cc:2258] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:41.673921 5885 ts_tablet_manager.cc:1875] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:41.675655 5885 ts_tablet_manager.cc:1888] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.275
I20240501 07:59:41.675719 5885 log.cc:1196] T 2fcfbbe5535f45ebad6733432d0f284e P e2c508eb3dac4193a1eeba799e5d54f2: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/2fcfbbe5535f45ebad6733432d0f284e
I20240501 07:59:41.676218 2253 catalog_manager.cc:4830] TS e2c508eb3dac4193a1eeba799e5d54f2 (127.0.99.133:35591): tablet 2fcfbbe5535f45ebad6733432d0f284e (table rebalance_test_table_0 [id=b69d178630d34f13b7c00a0f739d1d9e]) successfully deleted
I20240501 07:59:41.994457 2670 consensus_queue.cc:237] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 192, Committed index: 192, Last appended: 1.192, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 193 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } }
I20240501 07:59:41.995891 2931 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 192. Preceding OpId from leader: term: 1 index: 193. (index mismatch)
I20240501 07:59:41.995898 2540 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 192. Preceding OpId from leader: term: 1 index: 193. (index mismatch)
I20240501 07:59:41.995898 5001 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 192. Preceding OpId from leader: term: 1 index: 193. (index mismatch)
I20240501 07:59:41.996161 2796 raft_consensus.cc:1261] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Refusing update from remote peer 5dd2fd1701284a6b886ab8d14d1bd8df: Log matching property violated. Preceding OpId in replica: term: 1 index: 192. Preceding OpId from leader: term: 1 index: 193. (index mismatch)
I20240501 07:59:41.996345 5825 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 193, Last known committed idx: 192, Time since last communication: 0.000s
I20240501 07:59:41.996556 5901 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 193, Last known committed idx: 192, Time since last communication: 0.000s
I20240501 07:59:41.996742 5836 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 193, Last known committed idx: 192, Time since last communication: 0.000s
I20240501 07:59:41.996876 5823 consensus_queue.cc:1035] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 193, Last known committed idx: 192, Time since last communication: 0.000s
I20240501 07:59:41.998656 5901 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df [term 1 LEADER]: Committing config change with OpId 1.193: config changed from index 192 to 193, VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) evicted. New config: { opid_index: 193 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.999135 2931 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P e2c508eb3dac4193a1eeba799e5d54f2 [term 1 FOLLOWER]: Committing config change with OpId 1.193: config changed from index 192 to 193, VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) evicted. New config: { opid_index: 193 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:41.999778 5001 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 72d590cc1b1c4fbc86fc4b340d0c484a [term 1 FOLLOWER]: Committing config change with OpId 1.193: config changed from index 192 to 193, VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) evicted. New config: { opid_index: 193 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:42.000459 2252 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5414588c25d048ed84ec9aae6f300ddf with cas_config_opid_index 192: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:42.000919 2540 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 5810fd56ce9244438edcd3c0ea23e72c [term 1 FOLLOWER]: Committing config change with OpId 1.193: config changed from index 192 to 193, VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) evicted. New config: { opid_index: 193 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:42.001112 2796 raft_consensus.cc:2932] T 5414588c25d048ed84ec9aae6f300ddf P 3bd6898f790b48d593253180b79c5e40 [term 1 FOLLOWER]: Committing config change with OpId 1.193: config changed from index 192 to 193, VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) evicted. New config: { opid_index: 193 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } } }
I20240501 07:59:42.002701 2264 catalog_manager.cc:5484] T 5414588c25d048ed84ec9aae6f300ddf P 5dd2fd1701284a6b886ab8d14d1bd8df reported cstate change: config changed from index 192 to 193, VOTER e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" committed_config { opid_index: 193 OBSOLETE_local: false peers { permanent_uuid: "5dd2fd1701284a6b886ab8d14d1bd8df" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bd6898f790b48d593253180b79c5e40" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2c508eb3dac4193a1eeba799e5d54f2" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 35591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5810fd56ce9244438edcd3c0ea23e72c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 32773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72d590cc1b1c4fbc86fc4b340d0c484a" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:42.008929 3850 tablet_service.cc:1514] Processing DeleteTablet for tablet 5414588c25d048ed84ec9aae6f300ddf with delete_type TABLET_DATA_TOMBSTONED (TS e48105f8f6ab476196dd66c4b12808d8 not found in new config with opid_index 193) from {username='slave'} at 127.0.0.1:59124
I20240501 07:59:42.009428 5904 tablet_replica.cc:330] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: stopping tablet replica
I20240501 07:59:42.009764 5904 raft_consensus.cc:2229] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:42.010016 5904 raft_consensus.cc:2258] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:42.011512 5904 ts_tablet_manager.cc:1875] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:42.013484 5904 ts_tablet_manager.cc:1888] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.192
I20240501 07:59:42.013572 5904 log.cc:1196] T 5414588c25d048ed84ec9aae6f300ddf P e48105f8f6ab476196dd66c4b12808d8: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/5414588c25d048ed84ec9aae6f300ddf
I20240501 07:59:42.014112 2253 catalog_manager.cc:4830] TS e48105f8f6ab476196dd66c4b12808d8 (127.0.99.135:37389): tablet 5414588c25d048ed84ec9aae6f300ddf (table rebalance_test_table_4 [id=75c1926e3eaa46488929571348882819]) successfully deleted
I20240501 07:59:42.167456 2520 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:42.168917 3043 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 07:59:42.179781 2649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 07:59:42.186857 2782 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:42.190835 3850 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:42.194881 2912 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:42.195711 2387 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:42.199772 4982 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:42.322592 3043 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 07:59:42.324550 2649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 07:59:42.329548 2520 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:42.330976 2782 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:42.343703 2912 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 07:59:42.345705 4982 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:42.350528 3850 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:42.351689 2387 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:42.477985 2520 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:42.478515 2649 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 07:59:42.479518 3043 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 07:59:42.482267 2782 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 07:59:42.483150 4982 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:42.484665 2387 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 07:59:42.507763 3850 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:42.518819 2912 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
9c220cab46b743088853d0027749db0d | 127.0.99.190:35233 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:36633 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
1537338fbeea47ee81f7a210cd6df2f0 | 127.0.99.134:44027 | HEALTHY | <none> | 2 | 0
3bd6898f790b48d593253180b79c5e40 | 127.0.99.132:40829 | HEALTHY | <none> | 1 | 0
5810fd56ce9244438edcd3c0ea23e72c | 127.0.99.130:32773 | HEALTHY | <none> | 1 | 0
5dd2fd1701284a6b886ab8d14d1bd8df | 127.0.99.131:44519 | HEALTHY | <none> | 4 | 0
72d590cc1b1c4fbc86fc4b340d0c484a | 127.0.99.136:45877 | HEALTHY | <none> | 0 | 0
85377a976c05408fbb384237a0edb691 | 127.0.99.129:42933 | HEALTHY | <none> | 3 | 0
e2c508eb3dac4193a1eeba799e5d54f2 | 127.0.99.133:35591 | HEALTHY | <none> | 5 | 0
e48105f8f6ab476196dd66c4b12808d8 | 127.0.99.135:37389 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 8
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 8 server(s) checked
local_ip_for_outbound_sockets | 127.0.99.129 | experimental | 127.0.99.129:42933
local_ip_for_outbound_sockets | 127.0.99.130 | experimental | 127.0.99.130:32773
local_ip_for_outbound_sockets | 127.0.99.131 | experimental | 127.0.99.131:44519
local_ip_for_outbound_sockets | 127.0.99.132 | experimental | 127.0.99.132:40829
local_ip_for_outbound_sockets | 127.0.99.133 | experimental | 127.0.99.133:35591
local_ip_for_outbound_sockets | 127.0.99.134 | experimental | 127.0.99.134:44027
local_ip_for_outbound_sockets | 127.0.99.135 | experimental | 127.0.99.135:37389
local_ip_for_outbound_sockets | 127.0.99.136 | experimental | 127.0.99.136:45877
never_fsync | true | unsafe,advanced | all 8 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 8 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 8 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 8 server(s) checked
server_dump_info_format | pb | hidden | all 8 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.0.99.129:42933
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.0.99.130:32773
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.0.99.131:44519
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.0.99.132:40829
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.0.99.133:35591
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.0.99.134:44027
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.0.99.135:37389
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.0.99.136:45877
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:36633 | all 8 server(s) checked
time_source | builtin | all 8 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 9 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 5 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 10
First Quartile | 10
Median | 10
Third Quartile | 10
Maximum | 10
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 8
Tables | 6
Tablets | 16
Replicas | 80
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240501 07:59:42.542284 398 log_verifier.cc:126] Checking tablet 1ae5c568fb854f2a9e4122fc21314024
I20240501 07:59:42.638906 398 log_verifier.cc:177] Verified matching terms for 182 ops in tablet 1ae5c568fb854f2a9e4122fc21314024
I20240501 07:59:42.639060 398 log_verifier.cc:126] Checking tablet 2a62d5460d7844ebb51f4b33f915bd74
I20240501 07:59:42.733515 398 log_verifier.cc:177] Verified matching terms for 203 ops in tablet 2a62d5460d7844ebb51f4b33f915bd74
I20240501 07:59:42.733721 398 log_verifier.cc:126] Checking tablet 2fcfbbe5535f45ebad6733432d0f284e
I20240501 07:59:42.860512 398 log_verifier.cc:177] Verified matching terms for 276 ops in tablet 2fcfbbe5535f45ebad6733432d0f284e
I20240501 07:59:42.860716 398 log_verifier.cc:126] Checking tablet 382e9b7922f849f980b80ded29e99075
I20240501 07:59:42.948158 398 log_verifier.cc:177] Verified matching terms for 181 ops in tablet 382e9b7922f849f980b80ded29e99075
I20240501 07:59:42.948339 398 log_verifier.cc:126] Checking tablet 3ade42f3f20e4deda01e89ee6ee4141f
I20240501 07:59:43.037560 398 log_verifier.cc:177] Verified matching terms for 179 ops in tablet 3ade42f3f20e4deda01e89ee6ee4141f
I20240501 07:59:43.037710 398 log_verifier.cc:126] Checking tablet 490523aeb9e44f4ba8c2fc42fb73a95e
I20240501 07:59:43.129513 398 log_verifier.cc:177] Verified matching terms for 182 ops in tablet 490523aeb9e44f4ba8c2fc42fb73a95e
I20240501 07:59:43.129688 398 log_verifier.cc:126] Checking tablet 5414588c25d048ed84ec9aae6f300ddf
I20240501 07:59:43.227022 398 log_verifier.cc:177] Verified matching terms for 193 ops in tablet 5414588c25d048ed84ec9aae6f300ddf
I20240501 07:59:43.227185 398 log_verifier.cc:126] Checking tablet 6949f70a1ae44576b42d388231f0b50f
I20240501 07:59:43.343137 398 log_verifier.cc:177] Verified matching terms for 259 ops in tablet 6949f70a1ae44576b42d388231f0b50f
I20240501 07:59:43.343389 398 log_verifier.cc:126] Checking tablet 902e7d877089496689096005187548fe
I20240501 07:59:43.454711 398 log_verifier.cc:177] Verified matching terms for 250 ops in tablet 902e7d877089496689096005187548fe
I20240501 07:59:43.454903 398 log_verifier.cc:126] Checking tablet 917c88788fbf4089a5df0f271cc478f7
I20240501 07:59:43.545048 398 log_verifier.cc:177] Verified matching terms for 185 ops in tablet 917c88788fbf4089a5df0f271cc478f7
I20240501 07:59:43.545215 398 log_verifier.cc:126] Checking tablet 97a598d6cbb84578ab82e3807b98f36f
I20240501 07:59:43.630098 398 log_verifier.cc:177] Verified matching terms for 153 ops in tablet 97a598d6cbb84578ab82e3807b98f36f
I20240501 07:59:43.630249 398 log_verifier.cc:126] Checking tablet a42ea1bbca294f3e9298d895eee087c7
I20240501 07:59:43.737329 398 log_verifier.cc:177] Verified matching terms for 226 ops in tablet a42ea1bbca294f3e9298d895eee087c7
I20240501 07:59:43.737501 398 log_verifier.cc:126] Checking tablet cb4496116793436f849ea95bcc6fc6ba
I20240501 07:59:43.764219 398 log_verifier.cc:177] Verified matching terms for 7 ops in tablet cb4496116793436f849ea95bcc6fc6ba
I20240501 07:59:43.764304 398 log_verifier.cc:126] Checking tablet ceea369d9dcb4a8c94ec88adde948ef6
I20240501 07:59:43.858224 398 log_verifier.cc:177] Verified matching terms for 182 ops in tablet ceea369d9dcb4a8c94ec88adde948ef6
I20240501 07:59:43.858398 398 log_verifier.cc:126] Checking tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa
I20240501 07:59:43.983248 398 log_verifier.cc:177] Verified matching terms for 274 ops in tablet cf2bcc9afe2f4c839e80fadb7fcf4dfa
I20240501 07:59:43.983474 398 log_verifier.cc:126] Checking tablet f64014c3e7474e21b65082390fc1e6a6
I20240501 07:59:44.099571 398 log_verifier.cc:177] Verified matching terms for 249 ops in tablet f64014c3e7474e21b65082390fc1e6a6
I20240501 07:59:44.101162 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 2327
I20240501 07:59:44.109305 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 2458
I20240501 07:59:44.116401 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 2589
I20240501 07:59:44.122701 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 2720
W20240501 07:59:44.127653 2868 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.99.129:42933: connect: Connection refused (error 111)
W20240501 07:59:44.128551 2868 consensus_peers.cc:485] T cb4496116793436f849ea95bcc6fc6ba P e2c508eb3dac4193a1eeba799e5d54f2 -> Peer 85377a976c05408fbb384237a0edb691 (127.0.99.129:42933): Couldn't send request to peer 85377a976c05408fbb384237a0edb691. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.129:42933: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:44.128813 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 2851
I20240501 07:59:44.135296 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 2982
I20240501 07:59:44.141348 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 3788
I20240501 07:59:44.146971 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 4920
I20240501 07:59:44.152276 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 2236
2024-05-01T07:59:44Z chronyd exiting
[ OK ] RebalanceParamTest.Rebalance/7 (30104 ms)
[----------] 1 test from RebalanceParamTest (30104 ms total)
[----------] 1 test from TserverAddedDuringRebalancingTest
[ RUN ] TserverAddedDuringRebalancingTest.TserverStarts/1
I20240501 07:59:44.234974 398 test_util.cc:268] Using random seed: -560328233
I20240501 07:59:44.235736 398 ts_itest-base.cc:112] Starting cluster with:
I20240501 07:59:44.235777 398 ts_itest-base.cc:113] --------------
I20240501 07:59:44.235787 398 ts_itest-base.cc:114] 8 tablet servers
I20240501 07:59:44.235796 398 ts_itest-base.cc:115] 3 replicas per TS
I20240501 07:59:44.235805 398 ts_itest-base.cc:116] --------------
2024-05-01T07:59:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T07:59:44Z Disabled control of system clock
I20240501 07:59:44.243674 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:41421
--webserver_interface=127.0.99.190
--webserver_port=0
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.99.190:41421
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240501 07:59:44.350350 6028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:44.350610 6028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:44.350648 6028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:44.353766 6028 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:44.353850 6028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 07:59:44.353873 6028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:44.353889 6028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 07:59:44.353909 6028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 07:59:44.357718 6028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.0.99.190:41421
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:41421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.99.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.6028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:44.358681 6028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:44.359793 6028 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:44.367812 6036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:44.368381 6033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:44.369073 6028 server_base.cc:1034] running on GCE node
W20240501 07:59:44.369463 6034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:44.370026 6028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:44.371024 6028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:44.372185 6028 hybrid_clock.cc:648] HybridClock initialized: now 1714550384372168 us; error 34 us; skew 500 ppm
I20240501 07:59:44.378338 6028 webserver.cc:460] Webserver started at http://127.0.99.190:43375/ using document root <none> and password file <none>
I20240501 07:59:44.378901 6028 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:44.378965 6028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:44.379128 6028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:44.380843 6028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "b65b89b6d76f4286ad3d493da5bb629c"
format_stamp: "Formatted at 2024-05-01 07:59:44 on dist-test-slave-f81v"
I20240501 07:59:44.381283 6028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "b65b89b6d76f4286ad3d493da5bb629c"
format_stamp: "Formatted at 2024-05-01 07:59:44 on dist-test-slave-f81v"
I20240501 07:59:44.384616 6028 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 07:59:44.386875 6042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:44.388020 6028 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20240501 07:59:44.388139 6028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
uuid: "b65b89b6d76f4286ad3d493da5bb629c"
format_stamp: "Formatted at 2024-05-01 07:59:44 on dist-test-slave-f81v"
I20240501 07:59:44.388232 6028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:44.425431 6028 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:44.426100 6028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:44.426256 6028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:44.432936 6028 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:41421
I20240501 07:59:44.432992 6094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:41421 every 8 connection(s)
I20240501 07:59:44.433945 6028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 07:59:44.436705 6095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:44.439268 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 6028
I20240501 07:59:44.439391 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 07:59:44.442143 6095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c: Bootstrap starting.
I20240501 07:59:44.444459 6095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:44.445247 6095 log.cc:824] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:44.447064 6095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c: No bootstrap required, opened a new log
I20240501 07:59:44.449818 6095 raft_consensus.cc:348] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b65b89b6d76f4286ad3d493da5bb629c" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 41421 } }
I20240501 07:59:44.450008 6095 raft_consensus.cc:374] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:44.450042 6095 raft_consensus.cc:729] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b65b89b6d76f4286ad3d493da5bb629c, State: Initialized, Role: FOLLOWER
I20240501 07:59:44.450394 6095 consensus_queue.cc:260] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b65b89b6d76f4286ad3d493da5bb629c" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 41421 } }
I20240501 07:59:44.450495 6095 raft_consensus.cc:388] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 07:59:44.450537 6095 raft_consensus.cc:482] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 07:59:44.450616 6095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:44.451341 6095 raft_consensus.cc:504] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b65b89b6d76f4286ad3d493da5bb629c" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 41421 } }
I20240501 07:59:44.451639 6095 leader_election.cc:302] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b65b89b6d76f4286ad3d493da5bb629c; no voters:
I20240501 07:59:44.451913 6095 leader_election.cc:288] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 07:59:44.452029 6100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:44.452183 6100 raft_consensus.cc:686] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [term 1 LEADER]: Becoming Leader. State: Replica: b65b89b6d76f4286ad3d493da5bb629c, State: Running, Role: LEADER
I20240501 07:59:44.452548 6100 consensus_queue.cc:237] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b65b89b6d76f4286ad3d493da5bb629c" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 41421 } }
I20240501 07:59:44.453090 6095 sys_catalog.cc:564] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [sys.catalog]: configured and running, proceeding with master startup.
I20240501 07:59:44.454156 6102 sys_catalog.cc:455] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b65b89b6d76f4286ad3d493da5bb629c. Latest consensus state: current_term: 1 leader_uuid: "b65b89b6d76f4286ad3d493da5bb629c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b65b89b6d76f4286ad3d493da5bb629c" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 41421 } } }
I20240501 07:59:44.454260 6102 sys_catalog.cc:458] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [sys.catalog]: This master's current role is: LEADER
I20240501 07:59:44.454527 6101 sys_catalog.cc:455] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b65b89b6d76f4286ad3d493da5bb629c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b65b89b6d76f4286ad3d493da5bb629c" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 41421 } } }
I20240501 07:59:44.454612 6101 sys_catalog.cc:458] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c [sys.catalog]: This master's current role is: LEADER
I20240501 07:59:44.454615 6109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 07:59:44.457430 6109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 07:59:44.462796 6109 catalog_manager.cc:1347] Generated new cluster ID: 9bd601cd313e4dacb95214f8756d1891
I20240501 07:59:44.462886 6109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 07:59:44.501582 6109 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 07:59:44.502848 6109 catalog_manager.cc:1504] Loading token signing keys...
I20240501 07:59:44.520378 6109 catalog_manager.cc:5857] T 00000000000000000000000000000000 P b65b89b6d76f4286ad3d493da5bb629c: Generated new TSK 0
I20240501 07:59:44.521315 6109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 07:59:44.533865 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.129:0
--local_ip_for_outbound_sockets=127.0.99.129
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41421
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240501 07:59:44.642948 6119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:44.643204 6119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:44.643256 6119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:44.646365 6119 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:44.646454 6119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:44.646543 6119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.129
I20240501 07:59:44.650336 6119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.99.129
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.6119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.129
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:44.651304 6119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:44.652558 6119 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:44.661445 6127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:44.662086 6124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:44.662452 6125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:44.663084 6119 server_base.cc:1034] running on GCE node
I20240501 07:59:44.663529 6119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:44.664168 6119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:44.665313 6119 hybrid_clock.cc:648] HybridClock initialized: now 1714550384665295 us; error 38 us; skew 500 ppm
I20240501 07:59:44.667265 6119 webserver.cc:460] Webserver started at http://127.0.99.129:35425/ using document root <none> and password file <none>
I20240501 07:59:44.667819 6119 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:44.667873 6119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:44.668087 6119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:44.669852 6119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
format_stamp: "Formatted at 2024-05-01 07:59:44 on dist-test-slave-f81v"
I20240501 07:59:44.670336 6119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
format_stamp: "Formatted at 2024-05-01 07:59:44 on dist-test-slave-f81v"
I20240501 07:59:44.673825 6119 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 07:59:44.676146 6133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:44.677181 6119 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 07:59:44.677300 6119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
format_stamp: "Formatted at 2024-05-01 07:59:44 on dist-test-slave-f81v"
I20240501 07:59:44.677394 6119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:44.697736 6119 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:44.698380 6119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:44.698545 6119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:44.699060 6119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:44.699993 6119 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:44.700039 6119 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:44.700093 6119 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:44.700119 6119 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:44.708293 6119 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:44189
I20240501 07:59:44.708328 6246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:44189 every 8 connection(s)
I20240501 07:59:44.709229 6119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 07:59:44.710069 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 6119
I20240501 07:59:44.710175 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 07:59:44.712333 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:0
--local_ip_for_outbound_sockets=127.0.99.130
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41421
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 07:59:44.718679 6247 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:44.718943 6247 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:44.719506 6247 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:44.721062 6059 ts_manager.cc:194] Registered new tserver with Master: 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:44.722218 6059 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.129:35673
W20240501 07:59:44.821058 6250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:44.821300 6250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:44.821367 6250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:44.824445 6250 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:44.824520 6250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:44.824611 6250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240501 07:59:44.828338 6250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.6250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:44.829282 6250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:44.830410 6250 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:44.839103 6255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:44.839584 6258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:44.840108 6256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:44.840771 6250 server_base.cc:1034] running on GCE node
I20240501 07:59:44.841327 6250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:44.841919 6250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:44.843072 6250 hybrid_clock.cc:648] HybridClock initialized: now 1714550384843057 us; error 31 us; skew 500 ppm
I20240501 07:59:44.844985 6250 webserver.cc:460] Webserver started at http://127.0.99.130:34877/ using document root <none> and password file <none>
I20240501 07:59:44.845522 6250 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:44.845577 6250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:44.845754 6250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:44.847450 6250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "4a155eaa444d440d90fde420951458e4"
format_stamp: "Formatted at 2024-05-01 07:59:44 on dist-test-slave-f81v"
I20240501 07:59:44.847957 6250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "4a155eaa444d440d90fde420951458e4"
format_stamp: "Formatted at 2024-05-01 07:59:44 on dist-test-slave-f81v"
I20240501 07:59:44.851294 6250 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:44.853610 6264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:44.854745 6250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:44.854852 6250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "4a155eaa444d440d90fde420951458e4"
format_stamp: "Formatted at 2024-05-01 07:59:44 on dist-test-slave-f81v"
I20240501 07:59:44.854952 6250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:44.875592 6250 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:44.876297 6250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:44.876461 6250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:44.876989 6250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:44.877907 6250 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:44.877954 6250 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:44.877997 6250 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:44.878026 6250 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:44.886312 6250 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:35285
I20240501 07:59:44.886343 6377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:35285 every 8 connection(s)
I20240501 07:59:44.887267 6250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 07:59:44.888810 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 6250
I20240501 07:59:44.888919 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 07:59:44.891098 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:0
--local_ip_for_outbound_sockets=127.0.99.131
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41421
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 07:59:44.897044 6378 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:44.897281 6378 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:44.897861 6378 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:44.898890 6059 ts_manager.cc:194] Registered new tserver with Master: 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:44.899449 6059 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:37893
W20240501 07:59:45.001675 6381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:45.001919 6381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:45.001972 6381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:45.005039 6381 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:45.005105 6381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:45.005184 6381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240501 07:59:45.008880 6381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.6381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:45.009780 6381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:45.010903 6381 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:45.019465 6389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:45.019959 6386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:45.020522 6387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:45.021106 6381 server_base.cc:1034] running on GCE node
I20240501 07:59:45.021533 6381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:45.022099 6381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:45.023244 6381 hybrid_clock.cc:648] HybridClock initialized: now 1714550385023219 us; error 40 us; skew 500 ppm
I20240501 07:59:45.025115 6381 webserver.cc:460] Webserver started at http://127.0.99.131:36913/ using document root <none> and password file <none>
I20240501 07:59:45.025655 6381 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:45.025714 6381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:45.025882 6381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:45.027535 6381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.028036 6381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.031208 6381 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20240501 07:59:45.033367 6395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.034401 6381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:45.034504 6381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.034591 6381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:45.049770 6381 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:45.050359 6381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:45.050509 6381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:45.051014 6381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:45.051915 6381 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:45.051990 6381 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.052052 6381 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:45.052078 6381 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.060576 6381 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:41487
I20240501 07:59:45.060600 6508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:41487 every 8 connection(s)
I20240501 07:59:45.061502 6381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 07:59:45.067071 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 6381
I20240501 07:59:45.067183 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 07:59:45.069231 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:0
--local_ip_for_outbound_sockets=127.0.99.132
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41421
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 07:59:45.070046 6509 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:45.070293 6509 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:45.070858 6509 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:45.071837 6059 ts_manager.cc:194] Registered new tserver with Master: 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 07:59:45.072417 6059 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:40547
W20240501 07:59:45.176285 6512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:45.176535 6512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:45.176600 6512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:45.179731 6512 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:45.179805 6512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:45.179895 6512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240501 07:59:45.183585 6512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.6512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:45.184579 6512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:45.185707 6512 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:45.194373 6517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:45.194799 6518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:45.195631 6520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:45.196166 6512 server_base.cc:1034] running on GCE node
I20240501 07:59:45.196593 6512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:45.197176 6512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:45.198316 6512 hybrid_clock.cc:648] HybridClock initialized: now 1714550385198299 us; error 37 us; skew 500 ppm
I20240501 07:59:45.200318 6512 webserver.cc:460] Webserver started at http://127.0.99.132:38617/ using document root <none> and password file <none>
I20240501 07:59:45.200863 6512 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:45.200918 6512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:45.201081 6512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:45.202811 6512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "d2e209265a41425fba98c9d9b0551bd4"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.203253 6512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "d2e209265a41425fba98c9d9b0551bd4"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.206625 6512 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 07:59:45.208914 6526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.209918 6512 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 07:59:45.210021 6512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "d2e209265a41425fba98c9d9b0551bd4"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.210129 6512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:45.221899 6512 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:45.222576 6512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:45.222726 6512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:45.223258 6512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:45.224300 6512 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:45.224346 6512 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.224398 6512 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:45.224422 6512 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.232658 6512 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:42501
I20240501 07:59:45.232678 6639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:42501 every 8 connection(s)
I20240501 07:59:45.233611 6512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 07:59:45.234901 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 6512
I20240501 07:59:45.235003 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 07:59:45.237211 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:0
--local_ip_for_outbound_sockets=127.0.99.133
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41421
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 07:59:45.243131 6640 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:45.243356 6640 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:45.243903 6640 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:45.244938 6059 ts_manager.cc:194] Registered new tserver with Master: d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:45.245483 6059 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:52369
W20240501 07:59:45.344698 6643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:45.344949 6643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:45.345017 6643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:45.348169 6643 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:45.348253 6643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:45.348345 6643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240501 07:59:45.351994 6643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.6643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:45.352874 6643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:45.354053 6643 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:45.362617 6648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:45.363162 6649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:45.364221 6651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:45.364293 6643 server_base.cc:1034] running on GCE node
I20240501 07:59:45.364753 6643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:45.365320 6643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:45.366511 6643 hybrid_clock.cc:648] HybridClock initialized: now 1714550385366476 us; error 49 us; skew 500 ppm
I20240501 07:59:45.368577 6643 webserver.cc:460] Webserver started at http://127.0.99.133:39035/ using document root <none> and password file <none>
I20240501 07:59:45.369122 6643 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:45.369179 6643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:45.369361 6643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:45.371085 6643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "7a80bdedf813425faf7617b4b07c9f36"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.372133 6643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "7a80bdedf813425faf7617b4b07c9f36"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.375669 6643 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20240501 07:59:45.377876 6657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.378872 6643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240501 07:59:45.378989 6643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "7a80bdedf813425faf7617b4b07c9f36"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.379076 6643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:45.392854 6643 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:45.393481 6643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:45.393628 6643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:45.394141 6643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:45.395037 6643 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:45.395087 6643 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.395143 6643 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:45.395169 6643 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.403218 6643 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:39247
I20240501 07:59:45.403224 6770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:39247 every 8 connection(s)
I20240501 07:59:45.404244 6643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 07:59:45.412868 6771 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:45.412940 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 6643
I20240501 07:59:45.413038 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240501 07:59:45.413144 6771 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:45.413789 6771 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:45.414837 6059 ts_manager.cc:194] Registered new tserver with Master: 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247)
I20240501 07:59:45.415345 6059 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:57065
I20240501 07:59:45.416234 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.134:0
--local_ip_for_outbound_sockets=127.0.99.134
--webserver_interface=127.0.99.134
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41421
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240501 07:59:45.525787 6774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:45.526012 6774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:45.526059 6774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:45.529100 6774 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:45.529167 6774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:45.529237 6774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.134
I20240501 07:59:45.532867 6774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.0.99.134
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.6774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.134
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:45.533780 6774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:45.534940 6774 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:45.543787 6779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:45.544271 6782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:45.544870 6780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:45.545472 6774 server_base.cc:1034] running on GCE node
I20240501 07:59:45.545893 6774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:45.546496 6774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:45.547647 6774 hybrid_clock.cc:648] HybridClock initialized: now 1714550385547633 us; error 30 us; skew 500 ppm
I20240501 07:59:45.549697 6774 webserver.cc:460] Webserver started at http://127.0.99.134:37409/ using document root <none> and password file <none>
I20240501 07:59:45.550251 6774 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:45.550307 6774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:45.550485 6774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:45.552227 6774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "29afadb287c049d9bb777d674b3697ce"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.552692 6774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "29afadb287c049d9bb777d674b3697ce"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.556213 6774 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 07:59:45.558566 6788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.559726 6774 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:45.559840 6774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "29afadb287c049d9bb777d674b3697ce"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.559962 6774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:45.578186 6774 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:45.578933 6774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:45.579108 6774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:45.579681 6774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:45.580807 6774 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:45.580868 6774 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.580919 6774 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:45.580946 6774 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.588999 6774 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.134:46293
I20240501 07:59:45.589022 6901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.134:46293 every 8 connection(s)
I20240501 07:59:45.589918 6774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240501 07:59:45.591768 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 6774
I20240501 07:59:45.591876 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240501 07:59:45.594156 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.135:0
--local_ip_for_outbound_sockets=127.0.99.135
--webserver_interface=127.0.99.135
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41421
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 07:59:45.599236 6902 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:45.599452 6902 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:45.600011 6902 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:45.601009 6059 ts_manager.cc:194] Registered new tserver with Master: 29afadb287c049d9bb777d674b3697ce (127.0.99.134:46293)
I20240501 07:59:45.601773 6059 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.134:53245
W20240501 07:59:45.701766 6905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:45.702018 6905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:45.702073 6905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:45.705124 6905 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:45.705196 6905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:45.705267 6905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.135
I20240501 07:59:45.709201 6905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.0.99.135
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.6905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.135
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:45.710271 6905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:45.711665 6905 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:45.721072 6913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:45.721565 6910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:45.722075 6911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:45.722662 6905 server_base.cc:1034] running on GCE node
I20240501 07:59:45.723083 6905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:45.723692 6905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:45.724586 6247 heartbeater.cc:503] Master 127.0.99.190:41421 was elected leader, sending a full tablet report...
I20240501 07:59:45.724866 6905 hybrid_clock.cc:648] HybridClock initialized: now 1714550385724831 us; error 50 us; skew 500 ppm
I20240501 07:59:45.727178 6905 webserver.cc:460] Webserver started at http://127.0.99.135:43095/ using document root <none> and password file <none>
I20240501 07:59:45.727775 6905 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:45.727833 6905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:45.728042 6905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:45.729744 6905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "ef83010b77b441ffac2e41468a0a1bd1"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.730203 6905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "ef83010b77b441ffac2e41468a0a1bd1"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.733675 6905 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 07:59:45.736191 6919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.737213 6905 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 07:59:45.737326 6905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "ef83010b77b441ffac2e41468a0a1bd1"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.737422 6905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:45.763455 6905 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:45.764142 6905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:45.764292 6905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:45.764847 6905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:45.765683 6905 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:45.765727 6905 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.765764 6905 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:45.765784 6905 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.774219 6905 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.135:34937
I20240501 07:59:45.774241 7032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.135:34937 every 8 connection(s)
I20240501 07:59:45.775228 6905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 07:59:45.780743 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 6905
I20240501 07:59:45.780900 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240501 07:59:45.783025 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.136:0
--local_ip_for_outbound_sockets=127.0.99.136
--webserver_interface=127.0.99.136
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41421
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 07:59:45.784373 7033 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:45.784658 7033 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:45.785306 7033 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:45.786383 6059 ts_manager.cc:194] Registered new tserver with Master: ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937)
I20240501 07:59:45.786921 6059 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.135:35985
W20240501 07:59:45.898165 7036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:45.898384 7036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:45.898442 7036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240501 07:59:45.901546 6378 heartbeater.cc:503] Master 127.0.99.190:41421 was elected leader, sending a full tablet report...
W20240501 07:59:45.901621 7036 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:45.901674 7036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:45.901777 7036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.136
I20240501 07:59:45.905678 7036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.136:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.0.99.136
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.7036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.136
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:45.906584 7036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:45.907706 7036 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:45.916260 7041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:45.916707 7042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:45.917260 7044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:45.917878 7036 server_base.cc:1034] running on GCE node
I20240501 07:59:45.918314 7036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:45.918898 7036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:45.920027 7036 hybrid_clock.cc:648] HybridClock initialized: now 1714550385920005 us; error 34 us; skew 500 ppm
I20240501 07:59:45.922116 7036 webserver.cc:460] Webserver started at http://127.0.99.136:36805/ using document root <none> and password file <none>
I20240501 07:59:45.922683 7036 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:45.922737 7036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:45.922906 7036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:45.924693 7036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "06c63579c7234194b5f6ef457dd5e3f8"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.925158 7036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "06c63579c7234194b5f6ef457dd5e3f8"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.928561 7036 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 07:59:45.930841 7050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.932019 7036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 07:59:45.932132 7036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "06c63579c7234194b5f6ef457dd5e3f8"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:45.932236 7036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:45.958703 7036 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:45.959311 7036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:45.959470 7036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:45.960072 7036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:45.960985 7036 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:45.961032 7036 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.961076 7036 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:45.961103 7036 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:45.969718 7036 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.136:40449
I20240501 07:59:45.969744 7163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.136:40449 every 8 connection(s)
I20240501 07:59:45.970661 7036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 07:59:45.979182 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 7036
I20240501 07:59:45.979317 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240501 07:59:45.979454 7164 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:45.979679 7164 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:45.980335 7164 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:45.981335 6055 ts_manager.cc:194] Registered new tserver with Master: 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136:40449)
I20240501 07:59:45.981909 6055 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.136:50483
I20240501 07:59:45.993497 398 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240501 07:59:46.002473 398 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240501 07:59:46.002553 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 6643
I20240501 07:59:46.007251 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 6774
I20240501 07:59:46.011787 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 6905
I20240501 07:59:46.018203 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 7036
I20240501 07:59:46.074838 6509 heartbeater.cc:503] Master 127.0.99.190:41421 was elected leader, sending a full tablet report...
I20240501 07:59:46.247437 6640 heartbeater.cc:503] Master 127.0.99.190:41421 was elected leader, sending a full tablet report...
I20240501 07:59:49.774353 6055 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43232:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:49.794404 6443 tablet_service.cc:1467] Processing CreateTablet for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:49.794417 6312 tablet_service.cc:1467] Processing CreateTablet for tablet 3098236442b242b38c5937de465c8e1f (DEFAULT_TABLE table=rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:49.795493 6312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3098236442b242b38c5937de465c8e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:49.795511 6443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:49.796309 6442 tablet_service.cc:1467] Processing CreateTablet for tablet 3098236442b242b38c5937de465c8e1f (DEFAULT_TABLE table=rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:49.796582 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3098236442b242b38c5937de465c8e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:49.798007 6180 tablet_service.cc:1467] Processing CreateTablet for tablet 71bcfcbbdfc0491ea936168dd699fc92 (DEFAULT_TABLE table=rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:49.798759 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71bcfcbbdfc0491ea936168dd699fc92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:49.799957 6176 tablet_service.cc:1467] Processing CreateTablet for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:49.800182 6176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:49.800220 6311 tablet_service.cc:1467] Processing CreateTablet for tablet 71bcfcbbdfc0491ea936168dd699fc92 (DEFAULT_TABLE table=rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:49.800482 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71bcfcbbdfc0491ea936168dd699fc92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:49.802259 7184 tablet_bootstrap.cc:492] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:49.802477 6181 tablet_service.cc:1467] Processing CreateTablet for tablet 3098236442b242b38c5937de465c8e1f (DEFAULT_TABLE table=rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:49.802721 6181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3098236442b242b38c5937de465c8e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:49.804847 7184 tablet_bootstrap.cc:654] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:49.805706 7184 log.cc:824] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:49.806645 7185 tablet_bootstrap.cc:492] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:49.807117 7187 tablet_bootstrap.cc:492] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:49.808595 7185 tablet_bootstrap.cc:654] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:49.809320 7185 log.cc:824] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:49.810137 7184 tablet_bootstrap.cc:492] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:49.809783 6573 tablet_service.cc:1467] Processing CreateTablet for tablet 71bcfcbbdfc0491ea936168dd699fc92 (DEFAULT_TABLE table=rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:49.810312 7187 tablet_bootstrap.cc:654] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:49.810436 7184 ts_tablet_manager.cc:1367] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20240501 07:59:49.809782 6574 tablet_service.cc:1467] Processing CreateTablet for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:49.810734 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71bcfcbbdfc0491ea936168dd699fc92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:49.811033 7187 log.cc:824] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:49.811089 6574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:49.812629 7187 tablet_bootstrap.cc:492] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:49.812799 7187 ts_tablet_manager.cc:1367] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240501 07:59:49.812857 7185 tablet_bootstrap.cc:492] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:49.813148 7185 ts_tablet_manager.cc:1367] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240501 07:59:49.814524 7184 raft_consensus.cc:348] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.814924 7184 raft_consensus.cc:374] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:49.815019 7184 raft_consensus.cc:729] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:49.815536 7184 consensus_queue.cc:260] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.816326 7187 raft_consensus.cc:348] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.816514 7187 raft_consensus.cc:374] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:49.816569 7187 raft_consensus.cc:729] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:49.816486 7184 ts_tablet_manager.cc:1398] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 07:59:49.816525 7185 raft_consensus.cc:348] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.816730 7185 raft_consensus.cc:374] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:49.816785 7185 raft_consensus.cc:729] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:49.816958 7187 consensus_queue.cc:260] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.817217 7185 consensus_queue.cc:260] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.817698 7187 ts_tablet_manager.cc:1398] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 07:59:49.818001 7187 tablet_bootstrap.cc:492] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:49.818854 7193 tablet_bootstrap.cc:492] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:49.818902 7187 tablet_bootstrap.cc:654] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:49.819628 7184 tablet_bootstrap.cc:492] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:49.820564 7185 ts_tablet_manager.cc:1398] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240501 07:59:49.820880 7193 tablet_bootstrap.cc:654] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:49.820891 7185 tablet_bootstrap.cc:492] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:49.821036 7184 tablet_bootstrap.cc:654] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:49.821682 7193 log.cc:824] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:49.821749 7185 tablet_bootstrap.cc:654] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:49.822190 7187 tablet_bootstrap.cc:492] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:49.822273 7187 ts_tablet_manager.cc:1367] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 07:59:49.823297 7193 tablet_bootstrap.cc:492] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:49.823460 7193 ts_tablet_manager.cc:1367] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240501 07:59:49.823577 7187 raft_consensus.cc:348] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.823673 7187 raft_consensus.cc:374] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:49.823704 7187 raft_consensus.cc:729] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:49.823825 7187 consensus_queue.cc:260] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.824107 7187 ts_tablet_manager.cc:1398] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:49.825461 7184 tablet_bootstrap.cc:492] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:49.825538 7184 ts_tablet_manager.cc:1367] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 07:59:49.825932 7185 tablet_bootstrap.cc:492] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:49.825955 7184 raft_consensus.cc:348] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.826018 7185 ts_tablet_manager.cc:1367] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 07:59:49.826056 7184 raft_consensus.cc:374] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:49.826093 7184 raft_consensus.cc:729] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:49.826220 7184 consensus_queue.cc:260] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.826454 7184 ts_tablet_manager.cc:1398] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:49.826514 7185 raft_consensus.cc:348] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.826634 7185 raft_consensus.cc:374] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:49.826664 7185 raft_consensus.cc:729] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:49.826795 7185 consensus_queue.cc:260] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.827085 7193 raft_consensus.cc:348] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.827275 7193 raft_consensus.cc:374] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:49.827323 7193 raft_consensus.cc:729] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:49.827708 7185 ts_tablet_manager.cc:1398] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:49.827775 7193 consensus_queue.cc:260] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.827879 7185 tablet_bootstrap.cc:492] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:49.828770 7193 ts_tablet_manager.cc:1398] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240501 07:59:49.828837 7185 tablet_bootstrap.cc:654] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:49.829141 7193 tablet_bootstrap.cc:492] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:49.830024 7185 tablet_bootstrap.cc:492] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:49.830063 7193 tablet_bootstrap.cc:654] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:49.830107 7185 ts_tablet_manager.cc:1367] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:49.830514 7185 raft_consensus.cc:348] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.830624 7185 raft_consensus.cc:374] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:49.830665 7185 raft_consensus.cc:729] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:49.830775 7185 consensus_queue.cc:260] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.831007 7185 ts_tablet_manager.cc:1398] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:49.831218 7193 tablet_bootstrap.cc:492] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:49.831303 7193 ts_tablet_manager.cc:1367] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:49.831743 7193 raft_consensus.cc:348] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.831821 7193 raft_consensus.cc:374] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:49.831840 7193 raft_consensus.cc:729] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:49.832010 7193 consensus_queue.cc:260] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.832278 7193 ts_tablet_manager.cc:1398] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240501 07:59:49.891613 6379 tablet.cc:2369] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 07:59:49.908982 7198 raft_consensus.cc:482] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:49.909179 7198 raft_consensus.cc:504] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.910236 7198 leader_election.cc:288] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:49.914234 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf00ed2af0e42ca94c99a701fbe7dd1" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:49.914333 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf00ed2af0e42ca94c99a701fbe7dd1" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:49.914582 6463 raft_consensus.cc:2454] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 0.
I20240501 07:59:49.914583 6201 raft_consensus.cc:2454] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 0.
I20240501 07:59:49.914971 6530 leader_election.cc:302] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:49.915195 7198 raft_consensus.cc:2781] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:49.915268 7198 raft_consensus.cc:482] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:49.915311 7198 raft_consensus.cc:3037] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:49.916188 7198 raft_consensus.cc:504] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.916627 7198 leader_election.cc:288] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:49.916858 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf00ed2af0e42ca94c99a701fbe7dd1" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:49.916898 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf00ed2af0e42ca94c99a701fbe7dd1" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:49.917006 6463 raft_consensus.cc:3037] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:49.917012 6201 raft_consensus.cc:3037] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:49.918099 6463 raft_consensus.cc:2454] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 1.
I20240501 07:59:49.918318 6201 raft_consensus.cc:2454] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 1.
I20240501 07:59:49.918370 6530 leader_election.cc:302] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:49.918529 7198 raft_consensus.cc:2781] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:49.918725 7198 raft_consensus.cc:686] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Becoming Leader. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Running, Role: LEADER
I20240501 07:59:49.919036 7198 consensus_queue.cc:237] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:49.921945 6055 catalog_manager.cc:5484] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:49.963785 6248 tablet.cc:2369] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 07:59:49.987716 6641 tablet.cc:2369] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 07:59:50.050873 7190 raft_consensus.cc:482] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:50.051086 7190 raft_consensus.cc:504] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:50.052232 7190 leader_election.cc:288] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:50.056245 6594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:50.056456 6594 raft_consensus.cc:2454] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:50.056874 6268 leader_election.cc:302] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:50.057116 7190 raft_consensus.cc:2781] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:50.057144 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:50.057212 7190 raft_consensus.cc:482] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:50.057264 7190 raft_consensus.cc:3037] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.057263 6201 raft_consensus.cc:2454] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:50.058141 7190 raft_consensus.cc:504] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:50.058480 7190 leader_election.cc:288] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Requested vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:50.058836 6594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:50.058910 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:50.058965 6594 raft_consensus.cc:3037] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.059026 6201 raft_consensus.cc:3037] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.059907 6201 raft_consensus.cc:2454] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:50.059986 6594 raft_consensus.cc:2454] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:50.060231 6268 leader_election.cc:302] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:50.060427 7190 raft_consensus.cc:2781] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:50.060653 7190 raft_consensus.cc:686] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Becoming Leader. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Running, Role: LEADER
I20240501 07:59:50.060961 7190 consensus_queue.cc:237] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:50.065359 6055 catalog_manager.cc:5484] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a155eaa444d440d90fde420951458e4 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:50.065865 6510 tablet.cc:2369] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 07:59:50.084940 7190 raft_consensus.cc:482] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:50.085078 7190 raft_consensus.cc:504] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:50.085603 7190 leader_election.cc:288] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:50.086387 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3098236442b242b38c5937de465c8e1f" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:50.086546 6201 raft_consensus.cc:2454] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:50.086875 6268 leader_election.cc:302] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:50.087060 7190 raft_consensus.cc:2781] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:50.087121 7190 raft_consensus.cc:482] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:50.087157 7190 raft_consensus.cc:3037] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.088172 7190 raft_consensus.cc:504] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:50.088544 7190 leader_election.cc:288] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:50.088891 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3098236442b242b38c5937de465c8e1f" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:50.089000 6201 raft_consensus.cc:3037] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.089318 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3098236442b242b38c5937de465c8e1f" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:50.089351 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3098236442b242b38c5937de465c8e1f" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:50.089461 6463 raft_consensus.cc:2454] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:50.089912 6201 raft_consensus.cc:2454] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:50.090169 6268 leader_election.cc:302] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 953b945e3e7e4c71a61aa99c1208ab1d; no voters: 5db2970f4bbd4a3c92d7d913f99716b1
I20240501 07:59:50.090317 7190 raft_consensus.cc:2781] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:50.090399 7190 raft_consensus.cc:686] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Becoming Leader. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Running, Role: LEADER
I20240501 07:59:50.090526 7190 consensus_queue.cc:237] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:50.092198 6055 catalog_manager.cc:5484] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a155eaa444d440d90fde420951458e4 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:50.266098 7210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:50.266350 7210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:50.269460 7210 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240501 07:59:50.358883 6594 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:50.359021 6201 raft_consensus.cc:1261] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:50.359316 6462 raft_consensus.cc:3037] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.359556 6200 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:50.359532 7190 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:50.360333 7207 consensus_queue.cc:1035] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:50.360634 7207 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:50.360687 6462 raft_consensus.cc:1261] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:50.360870 6200 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:50.361580 7207 consensus_queue.cc:1035] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:50.362754 7203 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:50.364248 6463 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:50.366035 7198 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:50.369657 7242 mvcc.cc:205] Tried to move back new op lower bound from 7022798398904938496 to 7022798397692088320. Current Snapshot: MvccSnapshot[applied={T|T < 7022798398904938496}]
I20240501 07:59:50.378633 7250 mvcc.cc:205] Tried to move back new op lower bound from 7022798398907633664 to 7022798397110546432. Current Snapshot: MvccSnapshot[applied={T|T < 7022798398907633664}]
I20240501 07:59:50.387718 7240 mvcc.cc:205] Tried to move back new op lower bound from 7022798398904938496 to 7022798397692088320. Current Snapshot: MvccSnapshot[applied={T|T < 7022798398904938496}]
INSERT report
rows total: 2000
time total: 95.6053 ms
time per row: 0.0478027 ms
I20240501 07:59:50.407624 7259 mvcc.cc:205] Tried to move back new op lower bound from 7022798398907633664 to 7022798397110546432. Current Snapshot: MvccSnapshot[applied={T|T < 7022798398907633664}]
I20240501 07:59:50.413722 6056 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43232:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:50.425580 6443 tablet_service.cc:1467] Processing CreateTablet for tablet f4c62996a5a64d978850f2f7a31009f4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:50.425769 6573 tablet_service.cc:1467] Processing CreateTablet for tablet f4c62996a5a64d978850f2f7a31009f4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:50.425812 6310 tablet_service.cc:1467] Processing CreateTablet for tablet 47d3021e3ac74c6ba482b8b54066c36e (DEFAULT_TABLE table=rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:50.425909 6443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c62996a5a64d978850f2f7a31009f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:50.425889 6176 tablet_service.cc:1467] Processing CreateTablet for tablet 47d3021e3ac74c6ba482b8b54066c36e (DEFAULT_TABLE table=rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:50.426012 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c62996a5a64d978850f2f7a31009f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:50.426093 6176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47d3021e3ac74c6ba482b8b54066c36e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:50.426102 6310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47d3021e3ac74c6ba482b8b54066c36e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:50.428207 6311 tablet_service.cc:1467] Processing CreateTablet for tablet f4c62996a5a64d978850f2f7a31009f4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:50.428229 6574 tablet_service.cc:1467] Processing CreateTablet for tablet 6f24f629734649a3b31c7e293122d7ce (DEFAULT_TABLE table=rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:50.428449 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c62996a5a64d978850f2f7a31009f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:50.428449 6574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f24f629734649a3b31c7e293122d7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:50.430464 6312 tablet_service.cc:1467] Processing CreateTablet for tablet 6f24f629734649a3b31c7e293122d7ce (DEFAULT_TABLE table=rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:50.430716 6312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f24f629734649a3b31c7e293122d7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:50.430794 7279 tablet_bootstrap.cc:492] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:50.431679 7279 tablet_bootstrap.cc:654] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:50.432379 7280 tablet_bootstrap.cc:492] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:50.433166 7280 tablet_bootstrap.cc:654] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:50.433207 6442 tablet_service.cc:1467] Processing CreateTablet for tablet 6f24f629734649a3b31c7e293122d7ce (DEFAULT_TABLE table=rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:50.433353 7279 tablet_bootstrap.cc:492] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:50.433445 7279 ts_tablet_manager.cc:1367] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:50.433485 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f24f629734649a3b31c7e293122d7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:50.433915 7279 raft_consensus.cc:348] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.434037 7279 raft_consensus.cc:374] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:50.434075 7279 raft_consensus.cc:729] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:50.434127 7280 tablet_bootstrap.cc:492] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:50.434214 7280 ts_tablet_manager.cc:1367] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:50.434221 7279 consensus_queue.cc:260] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.434501 7279 ts_tablet_manager.cc:1398] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:50.434660 7280 raft_consensus.cc:348] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.434762 7280 raft_consensus.cc:374] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:50.434793 7280 raft_consensus.cc:729] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:50.434893 7280 consensus_queue.cc:260] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.435133 7280 ts_tablet_manager.cc:1398] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:50.435319 7280 tablet_bootstrap.cc:492] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:50.436105 7280 tablet_bootstrap.cc:654] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:50.437469 7280 tablet_bootstrap.cc:492] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:50.437620 7280 ts_tablet_manager.cc:1367] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:50.437966 7280 raft_consensus.cc:348] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:50.438051 7280 raft_consensus.cc:374] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:50.438071 7280 raft_consensus.cc:729] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:50.438273 7280 consensus_queue.cc:260] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:50.438824 7280 ts_tablet_manager.cc:1398] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:50.439061 7280 tablet_bootstrap.cc:492] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:50.439836 7280 tablet_bootstrap.cc:654] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:50.433203 6441 tablet_service.cc:1467] Processing CreateTablet for tablet 47d3021e3ac74c6ba482b8b54066c36e (DEFAULT_TABLE table=rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:50.440367 6441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47d3021e3ac74c6ba482b8b54066c36e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:50.440757 7280 tablet_bootstrap.cc:492] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:50.440810 7280 ts_tablet_manager.cc:1367] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 07:59:50.441112 7280 raft_consensus.cc:348] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.441177 7280 raft_consensus.cc:374] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:50.441195 7280 raft_consensus.cc:729] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:50.441289 7280 consensus_queue.cc:260] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.441526 7280 ts_tablet_manager.cc:1398] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:50.443310 7278 tablet_bootstrap.cc:492] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:50.444269 7278 tablet_bootstrap.cc:654] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:50.444965 7277 tablet_bootstrap.cc:492] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:50.445849 7277 tablet_bootstrap.cc:654] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:50.446668 7278 tablet_bootstrap.cc:492] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:50.446753 7278 ts_tablet_manager.cc:1367] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 07:59:50.447171 7278 raft_consensus.cc:348] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:50.447278 7278 raft_consensus.cc:374] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:50.447302 7278 raft_consensus.cc:729] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:50.447407 7278 consensus_queue.cc:260] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:50.447978 7278 ts_tablet_manager.cc:1398] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:50.448150 7278 tablet_bootstrap.cc:492] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:50.448621 7277 tablet_bootstrap.cc:492] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:50.448709 7277 ts_tablet_manager.cc:1367] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 07:59:50.448978 7278 tablet_bootstrap.cc:654] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:50.449152 7277 raft_consensus.cc:348] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:50.449266 7277 raft_consensus.cc:374] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:50.449304 7277 raft_consensus.cc:729] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:50.449436 7277 consensus_queue.cc:260] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:50.449682 7277 ts_tablet_manager.cc:1398] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:50.450053 7277 tablet_bootstrap.cc:492] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:50.450413 7278 tablet_bootstrap.cc:492] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:50.450469 7278 ts_tablet_manager.cc:1367] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 07:59:50.451020 7277 tablet_bootstrap.cc:654] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:50.450990 7278 raft_consensus.cc:348] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.451092 7278 raft_consensus.cc:374] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:50.451139 7278 raft_consensus.cc:729] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:50.451262 7278 consensus_queue.cc:260] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.451802 7278 ts_tablet_manager.cc:1398] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 07:59:50.451974 7278 tablet_bootstrap.cc:492] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:50.452203 7277 tablet_bootstrap.cc:492] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:50.452327 7277 ts_tablet_manager.cc:1367] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:50.452687 7278 tablet_bootstrap.cc:654] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:50.452855 7277 raft_consensus.cc:348] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.453022 7277 raft_consensus.cc:374] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:50.453104 7277 raft_consensus.cc:729] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:50.453269 7277 consensus_queue.cc:260] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.454026 7277 ts_tablet_manager.cc:1398] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:50.454032 7278 tablet_bootstrap.cc:492] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:50.454115 7278 ts_tablet_manager.cc:1367] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:50.454545 7278 raft_consensus.cc:348] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.454643 7278 raft_consensus.cc:374] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:50.454680 7278 raft_consensus.cc:729] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:50.454809 7278 consensus_queue.cc:260] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.455070 7278 ts_tablet_manager.cc:1398] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:50.569931 7190 raft_consensus.cc:482] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:50.570068 7190 raft_consensus.cc:504] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:50.570469 7190 leader_election.cc:288] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:50.571012 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c62996a5a64d978850f2f7a31009f4" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:50.571022 6594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c62996a5a64d978850f2f7a31009f4" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:50.571172 6462 raft_consensus.cc:2454] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:50.571171 6594 raft_consensus.cc:2454] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:50.571456 6268 leader_election.cc:302] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:50.571642 7190 raft_consensus.cc:2781] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:50.571698 7190 raft_consensus.cc:482] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:50.571723 7190 raft_consensus.cc:3037] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.572531 7190 raft_consensus.cc:504] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:50.572852 7190 leader_election.cc:288] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:50.573225 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c62996a5a64d978850f2f7a31009f4" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:50.573264 6594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c62996a5a64d978850f2f7a31009f4" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:50.573339 6462 raft_consensus.cc:3037] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.573364 6594 raft_consensus.cc:3037] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.574081 6594 raft_consensus.cc:2454] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:50.574249 6462 raft_consensus.cc:2454] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:50.574342 6268 leader_election.cc:302] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:50.574492 7190 raft_consensus.cc:2781] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:50.574580 7190 raft_consensus.cc:686] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Becoming Leader. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Running, Role: LEADER
I20240501 07:59:50.574713 7190 consensus_queue.cc:237] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:50.576284 6056 catalog_manager.cc:5484] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a155eaa444d440d90fde420951458e4 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:50.603965 7258 raft_consensus.cc:482] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:50.604120 7258 raft_consensus.cc:504] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.605238 7258 leader_election.cc:288] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:50.609073 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d3021e3ac74c6ba482b8b54066c36e" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:50.609051 6332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d3021e3ac74c6ba482b8b54066c36e" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:50.609222 6201 raft_consensus.cc:2454] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:50.609269 6332 raft_consensus.cc:2454] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:50.609601 6399 leader_election.cc:302] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:50.609817 7258 raft_consensus.cc:2781] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:50.609879 7258 raft_consensus.cc:482] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:50.609918 7258 raft_consensus.cc:3037] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.610703 7258 raft_consensus.cc:504] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.611002 7258 leader_election.cc:288] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Requested vote from peers 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:50.611349 6332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d3021e3ac74c6ba482b8b54066c36e" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:50.611371 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d3021e3ac74c6ba482b8b54066c36e" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:50.611477 6201 raft_consensus.cc:3037] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.611519 6332 raft_consensus.cc:3037] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.612422 6201 raft_consensus.cc:2454] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:50.612422 6332 raft_consensus.cc:2454] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:50.612723 6399 leader_election.cc:302] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:50.612877 7258 raft_consensus.cc:2781] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:50.612988 7258 raft_consensus.cc:686] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Becoming Leader. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Running, Role: LEADER
I20240501 07:59:50.613281 7258 consensus_queue.cc:237] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.615689 6056 catalog_manager.cc:5484] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:50.751518 7258 raft_consensus.cc:482] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:50.751668 7258 raft_consensus.cc:504] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.752341 7258 leader_election.cc:288] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:50.753223 6332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f24f629734649a3b31c7e293122d7ce" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:50.753389 6332 raft_consensus.cc:2454] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:50.753716 6399 leader_election.cc:302] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:50.753924 7258 raft_consensus.cc:2781] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:50.753991 7258 raft_consensus.cc:482] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:50.754035 7258 raft_consensus.cc:3037] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.755070 7258 raft_consensus.cc:504] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.755468 7258 leader_election.cc:288] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Requested vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:50.755875 6332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f24f629734649a3b31c7e293122d7ce" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:50.756027 6332 raft_consensus.cc:3037] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:50.756337 6594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f24f629734649a3b31c7e293122d7ce" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:50.756381 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f24f629734649a3b31c7e293122d7ce" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:50.756469 6594 raft_consensus.cc:2454] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:50.756978 6332 raft_consensus.cc:2454] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:50.757247 6399 leader_election.cc:302] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters: d2e209265a41425fba98c9d9b0551bd4
I20240501 07:59:50.757486 7258 raft_consensus.cc:2781] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:50.757557 7258 raft_consensus.cc:686] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Becoming Leader. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Running, Role: LEADER
I20240501 07:59:50.757714 7258 consensus_queue.cc:237] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:50.759317 6056 catalog_manager.cc:5484] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } }
W20240501 07:59:50.912427 7296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:50.912684 7296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:50.915745 7296 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240501 07:59:50.994267 6593 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:50.994352 6462 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:50.994825 7207 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:50.995133 7207 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:50.996181 6332 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:50.997838 6201 raft_consensus.cc:1261] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:50.998073 7258 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:50.999001 6332 raft_consensus.cc:1261] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:50.999418 6593 raft_consensus.cc:3037] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.000000 7246 consensus_queue.cc:1035] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:51.000320 7246 consensus_queue.cc:1035] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:51.000535 6593 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:51.003428 7246 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 101.081 ms
time per row: 0.0505407 ms
I20240501 07:59:51.058306 6057 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43232:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:51.063642 6574 tablet_service.cc:1467] Processing CreateTablet for tablet 2bbceded7f37434ca25b1068bdfc8ca1 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.063704 6176 tablet_service.cc:1467] Processing CreateTablet for tablet 2bbceded7f37434ca25b1068bdfc8ca1 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.063980 6574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bbceded7f37434ca25b1068bdfc8ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.064385 6312 tablet_service.cc:1467] Processing CreateTablet for tablet 2bbceded7f37434ca25b1068bdfc8ca1 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.064617 6312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bbceded7f37434ca25b1068bdfc8ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.064797 6180 tablet_service.cc:1467] Processing CreateTablet for tablet 5d0ae7c274af4ce1992e9507cbabc551 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.065160 6181 tablet_service.cc:1467] Processing CreateTablet for tablet 4e052973ae7744e8887d3df0d420f411 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.065690 6573 tablet_service.cc:1467] Processing CreateTablet for tablet 5d0ae7c274af4ce1992e9507cbabc551 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.065888 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d0ae7c274af4ce1992e9507cbabc551. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.066406 7355 tablet_bootstrap.cc:492] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:51.067251 7355 tablet_bootstrap.cc:654] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.067499 7356 tablet_bootstrap.cc:492] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:51.067651 6572 tablet_service.cc:1467] Processing CreateTablet for tablet 4e052973ae7744e8887d3df0d420f411 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.067858 6572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e052973ae7744e8887d3df0d420f411. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.068562 6312 tablet_service.cc:1467] Processing CreateTablet for tablet 4e052973ae7744e8887d3df0d420f411 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.068660 7356 tablet_bootstrap.cc:654] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.068771 6312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e052973ae7744e8887d3df0d420f411. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.069831 7356 tablet_bootstrap.cc:492] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:51.069911 7356 ts_tablet_manager.cc:1367] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:51.070272 7356 raft_consensus.cc:348] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.070484 7356 raft_consensus.cc:374] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.070566 7356 raft_consensus.cc:729] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.070724 7356 consensus_queue.cc:260] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.070740 6443 tablet_service.cc:1467] Processing CreateTablet for tablet 5d0ae7c274af4ce1992e9507cbabc551 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.070991 6443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d0ae7c274af4ce1992e9507cbabc551. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.071045 7356 ts_tablet_manager.cc:1398] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:51.071172 7356 tablet_bootstrap.cc:492] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:51.072667 6176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bbceded7f37434ca25b1068bdfc8ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.072667 7356 tablet_bootstrap.cc:654] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.073661 7356 tablet_bootstrap.cc:492] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:51.073745 7356 ts_tablet_manager.cc:1367] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:51.074071 7356 raft_consensus.cc:348] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.074143 7356 raft_consensus.cc:374] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.074162 7356 raft_consensus.cc:729] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.074257 7356 consensus_queue.cc:260] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.074499 7356 ts_tablet_manager.cc:1398] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:51.074959 6181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e052973ae7744e8887d3df0d420f411. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.076045 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d0ae7c274af4ce1992e9507cbabc551. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.076231 7360 tablet_bootstrap.cc:492] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:51.077021 7360 tablet_bootstrap.cc:654] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.077910 7360 tablet_bootstrap.cc:492] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:51.077986 7360 ts_tablet_manager.cc:1367] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:51.078331 7360 raft_consensus.cc:348] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.078414 7360 raft_consensus.cc:374] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.078442 7360 raft_consensus.cc:729] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.078577 7360 consensus_queue.cc:260] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.078809 7360 ts_tablet_manager.cc:1398] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:51.078964 7360 tablet_bootstrap.cc:492] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:51.080024 7360 tablet_bootstrap.cc:654] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.081248 7360 tablet_bootstrap.cc:492] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:51.081319 7360 ts_tablet_manager.cc:1367] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 07:59:51.081645 7360 raft_consensus.cc:348] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.081730 7360 raft_consensus.cc:374] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.081774 7360 raft_consensus.cc:729] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.081883 7360 consensus_queue.cc:260] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.082111 7360 ts_tablet_manager.cc:1398] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:51.082209 7363 tablet_bootstrap.cc:492] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:51.082266 7360 tablet_bootstrap.cc:492] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:51.083076 7360 tablet_bootstrap.cc:654] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.083081 7363 tablet_bootstrap.cc:654] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.083199 7355 tablet_bootstrap.cc:492] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:51.083281 7355 ts_tablet_manager.cc:1367] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240501 07:59:51.083667 7355 raft_consensus.cc:348] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.083753 7355 raft_consensus.cc:374] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.083777 7355 raft_consensus.cc:729] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.083879 7355 consensus_queue.cc:260] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.084250 7355 ts_tablet_manager.cc:1398] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:51.084401 7355 tablet_bootstrap.cc:492] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:51.084925 7363 tablet_bootstrap.cc:492] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:51.085000 7363 ts_tablet_manager.cc:1367] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:51.085060 7360 tablet_bootstrap.cc:492] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:51.085122 7360 ts_tablet_manager.cc:1367] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 07:59:51.085177 7355 tablet_bootstrap.cc:654] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.085417 7363 raft_consensus.cc:348] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.085511 7363 raft_consensus.cc:374] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.085479 7360 raft_consensus.cc:348] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.085541 7363 raft_consensus.cc:729] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.085568 7360 raft_consensus.cc:374] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.085595 7360 raft_consensus.cc:729] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.085721 7360 consensus_queue.cc:260] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.085683 7363 consensus_queue.cc:260] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.086062 7363 ts_tablet_manager.cc:1398] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:51.086375 7360 ts_tablet_manager.cc:1398] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:51.088217 7355 tablet_bootstrap.cc:492] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:51.088294 7355 ts_tablet_manager.cc:1367] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 07:59:51.088694 7355 raft_consensus.cc:348] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.088786 7355 raft_consensus.cc:374] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.088815 7355 raft_consensus.cc:729] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.088927 7355 consensus_queue.cc:260] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.089154 7355 ts_tablet_manager.cc:1398] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:51.089341 7355 tablet_bootstrap.cc:492] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:51.090097 7355 tablet_bootstrap.cc:654] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.090965 7355 tablet_bootstrap.cc:492] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:51.091027 7355 ts_tablet_manager.cc:1367] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:51.091387 7355 raft_consensus.cc:348] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.091490 7355 raft_consensus.cc:374] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.091529 7355 raft_consensus.cc:729] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.091646 7355 consensus_queue.cc:260] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.091881 7355 ts_tablet_manager.cc:1398] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:51.106938 7203 raft_consensus.cc:482] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:51.107074 7203 raft_consensus.cc:504] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.107568 7203 leader_election.cc:288] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:51.108016 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e052973ae7744e8887d3df0d420f411" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:51.108183 6201 raft_consensus.cc:2454] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 0.
I20240501 07:59:51.108527 6530 leader_election.cc:302] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 953b945e3e7e4c71a61aa99c1208ab1d, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:51.108708 7203 raft_consensus.cc:2781] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:51.108773 7203 raft_consensus.cc:482] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:51.108815 7203 raft_consensus.cc:3037] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.109697 7203 raft_consensus.cc:504] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.110062 7203 leader_election.cc:288] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 election: Requested vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:51.110402 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e052973ae7744e8887d3df0d420f411" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:51.110520 6201 raft_consensus.cc:3037] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.111299 6201 raft_consensus.cc:2454] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 1.
I20240501 07:59:51.111451 6332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e052973ae7744e8887d3df0d420f411" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:51.111528 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e052973ae7744e8887d3df0d420f411" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:51.111557 6530 leader_election.cc:302] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 953b945e3e7e4c71a61aa99c1208ab1d, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:51.111589 6332 raft_consensus.cc:2454] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 0.
I20240501 07:59:51.111714 7203 raft_consensus.cc:2781] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:51.111918 7203 raft_consensus.cc:686] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Becoming Leader. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Running, Role: LEADER
I20240501 07:59:51.112093 7203 consensus_queue.cc:237] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.113689 6057 catalog_manager.cc:5484] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:51.127142 7203 raft_consensus.cc:482] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:51.127295 7203 raft_consensus.cc:504] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.127708 7203 leader_election.cc:288] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:51.128173 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bbceded7f37434ca25b1068bdfc8ca1" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:51.128211 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bbceded7f37434ca25b1068bdfc8ca1" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:51.128331 6201 raft_consensus.cc:2454] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 0.
I20240501 07:59:51.128331 6331 raft_consensus.cc:2454] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 0.
I20240501 07:59:51.128619 6530 leader_election.cc:302] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:51.128875 7203 raft_consensus.cc:2781] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:51.128938 7203 raft_consensus.cc:482] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:51.128985 7203 raft_consensus.cc:3037] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.129776 7203 raft_consensus.cc:504] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.130084 7203 leader_election.cc:288] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 election: Requested vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:51.130458 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bbceded7f37434ca25b1068bdfc8ca1" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:51.130492 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bbceded7f37434ca25b1068bdfc8ca1" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:51.130587 6201 raft_consensus.cc:3037] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.130645 6331 raft_consensus.cc:3037] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.131299 6201 raft_consensus.cc:2454] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 1.
I20240501 07:59:51.131428 6331 raft_consensus.cc:2454] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 1.
I20240501 07:59:51.131634 6530 leader_election.cc:302] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 953b945e3e7e4c71a61aa99c1208ab1d, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:51.131803 7203 raft_consensus.cc:2781] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:51.131891 7203 raft_consensus.cc:686] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Becoming Leader. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Running, Role: LEADER
I20240501 07:59:51.132052 7203 consensus_queue.cc:237] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:51.133631 6057 catalog_manager.cc:5484] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:51.255200 7333 raft_consensus.cc:482] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:51.255340 7333 raft_consensus.cc:504] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.256434 7333 leader_election.cc:288] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:51.260187 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d0ae7c274af4ce1992e9507cbabc551" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:51.260321 6462 raft_consensus.cc:2454] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:51.260638 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d0ae7c274af4ce1992e9507cbabc551" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:51.260643 6137 leader_election.cc:302] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:51.260759 6593 raft_consensus.cc:2454] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:51.260841 7333 raft_consensus.cc:2781] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:51.260890 7333 raft_consensus.cc:482] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:51.260924 7333 raft_consensus.cc:3037] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.261693 7333 raft_consensus.cc:504] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.261962 7333 leader_election.cc:288] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:51.262238 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d0ae7c274af4ce1992e9507cbabc551" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:51.262358 6462 raft_consensus.cc:3037] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.262329 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d0ae7c274af4ce1992e9507cbabc551" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:51.262428 6593 raft_consensus.cc:3037] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.263325 6462 raft_consensus.cc:2454] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:51.263325 6593 raft_consensus.cc:2454] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:51.263602 6137 leader_election.cc:302] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:51.263804 7333 raft_consensus.cc:2781] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:51.264015 7333 raft_consensus.cc:686] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Becoming Leader. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Running, Role: LEADER
I20240501 07:59:51.264312 7333 consensus_queue.cc:237] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.266597 6057 catalog_manager.cc:5484] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:51.404027 7377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:51.404268 7377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:51.407331 7377 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240501 07:59:51.480146 6331 raft_consensus.cc:3037] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.481238 6331 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:51.481699 6201 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:51.482178 7369 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:51.482671 6201 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:51.483073 6462 raft_consensus.cc:1261] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:51.483108 7369 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:51.483156 6593 raft_consensus.cc:1261] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:51.483610 7375 consensus_queue.cc:1035] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:51.484238 7375 consensus_queue.cc:1035] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:51.488080 6331 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:51.488718 7369 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:51.489056 7369 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 92.2942 ms
time per row: 0.0461471 ms
I20240501 07:59:51.542220 6057 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43232:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:51.547355 6312 tablet_service.cc:1467] Processing CreateTablet for tablet d929e6e09be943b68061ad4b1bb8e9f1 (DEFAULT_TABLE table=rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.547389 6181 tablet_service.cc:1467] Processing CreateTablet for tablet d929e6e09be943b68061ad4b1bb8e9f1 (DEFAULT_TABLE table=rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.547675 6181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d929e6e09be943b68061ad4b1bb8e9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.547662 6312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d929e6e09be943b68061ad4b1bb8e9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.548451 6572 tablet_service.cc:1467] Processing CreateTablet for tablet 947b5d16224a42dcad709e7ddefc0b33 (DEFAULT_TABLE table=rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.548493 6573 tablet_service.cc:1467] Processing CreateTablet for tablet 8bf587ac1d8141e99fceab28d89e4ea9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.548700 6572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947b5d16224a42dcad709e7ddefc0b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.549331 6443 tablet_service.cc:1467] Processing CreateTablet for tablet d929e6e09be943b68061ad4b1bb8e9f1 (DEFAULT_TABLE table=rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.549575 6443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d929e6e09be943b68061ad4b1bb8e9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.549924 6311 tablet_service.cc:1467] Processing CreateTablet for tablet 8bf587ac1d8141e99fceab28d89e4ea9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.549957 6441 tablet_service.cc:1467] Processing CreateTablet for tablet 947b5d16224a42dcad709e7ddefc0b33 (DEFAULT_TABLE table=rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.550123 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bf587ac1d8141e99fceab28d89e4ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.550144 6441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947b5d16224a42dcad709e7ddefc0b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.551465 7363 tablet_bootstrap.cc:492] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:51.551769 6442 tablet_service.cc:1467] Processing CreateTablet for tablet 8bf587ac1d8141e99fceab28d89e4ea9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.551910 7355 tablet_bootstrap.cc:492] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:51.551982 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bf587ac1d8141e99fceab28d89e4ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.552294 7363 tablet_bootstrap.cc:654] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.552528 7355 tablet_bootstrap.cc:654] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.553267 7363 tablet_bootstrap.cc:492] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:51.553355 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bf587ac1d8141e99fceab28d89e4ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.553352 7363 ts_tablet_manager.cc:1367] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:51.553417 6181 tablet_service.cc:1467] Processing CreateTablet for tablet 947b5d16224a42dcad709e7ddefc0b33 (DEFAULT_TABLE table=rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:51.553656 6181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947b5d16224a42dcad709e7ddefc0b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:51.553733 7356 tablet_bootstrap.cc:492] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:51.553829 7363 raft_consensus.cc:348] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:51.553961 7363 raft_consensus.cc:374] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.553992 7363 raft_consensus.cc:729] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.554114 7363 consensus_queue.cc:260] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:51.554394 7363 ts_tablet_manager.cc:1398] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:51.554492 7356 tablet_bootstrap.cc:654] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.554559 7363 tablet_bootstrap.cc:492] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:51.554975 7360 tablet_bootstrap.cc:492] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:51.555274 7363 tablet_bootstrap.cc:654] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.555743 7360 tablet_bootstrap.cc:654] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.556174 7355 tablet_bootstrap.cc:492] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:51.556258 7355 ts_tablet_manager.cc:1367] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 07:59:51.556697 7355 raft_consensus.cc:348] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:51.556814 7355 raft_consensus.cc:374] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.556849 7355 raft_consensus.cc:729] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.556962 7355 consensus_queue.cc:260] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:51.557238 7355 ts_tablet_manager.cc:1398] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:51.557397 7355 tablet_bootstrap.cc:492] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:51.558360 7356 tablet_bootstrap.cc:492] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:51.558456 7356 ts_tablet_manager.cc:1367] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 07:59:51.558568 7355 tablet_bootstrap.cc:654] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.558809 7356 raft_consensus.cc:348] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:51.558908 7356 raft_consensus.cc:374] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.558949 7356 raft_consensus.cc:729] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.559055 7356 consensus_queue.cc:260] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:51.559314 7356 ts_tablet_manager.cc:1398] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:51.559480 7356 tablet_bootstrap.cc:492] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:51.559666 7363 tablet_bootstrap.cc:492] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:51.559763 7363 ts_tablet_manager.cc:1367] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 07:59:51.560194 7363 raft_consensus.cc:348] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:51.560297 7363 raft_consensus.cc:374] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.560334 7363 raft_consensus.cc:729] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.560448 7363 consensus_queue.cc:260] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:51.560699 7363 ts_tablet_manager.cc:1398] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:51.560860 7363 tablet_bootstrap.cc:492] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:51.560887 7360 tablet_bootstrap.cc:492] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:51.560976 7360 ts_tablet_manager.cc:1367] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 07:59:51.561326 7360 raft_consensus.cc:348] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:51.561414 7360 raft_consensus.cc:374] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.561441 7360 raft_consensus.cc:729] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.561561 7363 tablet_bootstrap.cc:654] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.561547 7360 consensus_queue.cc:260] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:51.561786 7360 ts_tablet_manager.cc:1398] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 07:59:51.561933 7360 tablet_bootstrap.cc:492] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:51.562682 7355 tablet_bootstrap.cc:492] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:51.562783 7355 ts_tablet_manager.cc:1367] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240501 07:59:51.562947 7360 tablet_bootstrap.cc:654] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.563156 7355 raft_consensus.cc:348] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.563252 7355 raft_consensus.cc:374] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.563287 7355 raft_consensus.cc:729] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.563405 7355 consensus_queue.cc:260] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.563669 7355 ts_tablet_manager.cc:1398] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:51.564213 7363 tablet_bootstrap.cc:492] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:51.564292 7363 ts_tablet_manager.cc:1367] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:51.564301 7356 tablet_bootstrap.cc:654] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:51.564729 7363 raft_consensus.cc:348] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.564832 7363 raft_consensus.cc:374] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.564864 7363 raft_consensus.cc:729] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.564963 7363 consensus_queue.cc:260] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.565291 7363 ts_tablet_manager.cc:1398] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:51.565580 7360 tablet_bootstrap.cc:492] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:51.565666 7360 ts_tablet_manager.cc:1367] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240501 07:59:51.566097 7360 raft_consensus.cc:348] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:51.566210 7360 raft_consensus.cc:374] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.566241 7360 raft_consensus.cc:729] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.566361 7360 consensus_queue.cc:260] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:51.566671 7356 tablet_bootstrap.cc:492] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:51.566690 7360 ts_tablet_manager.cc:1398] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 07:59:51.566772 7356 ts_tablet_manager.cc:1367] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240501 07:59:51.567166 7356 raft_consensus.cc:348] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.567276 7356 raft_consensus.cc:374] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:51.567308 7356 raft_consensus.cc:729] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:51.567425 7356 consensus_queue.cc:260] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.567705 7356 ts_tablet_manager.cc:1398] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:51.641983 7207 raft_consensus.cc:482] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:51.642112 7207 raft_consensus.cc:504] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.642491 7207 leader_election.cc:288] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:51.642930 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bf587ac1d8141e99fceab28d89e4ea9" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:51.642932 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bf587ac1d8141e99fceab28d89e4ea9" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:51.643075 6462 raft_consensus.cc:2454] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:51.643074 6593 raft_consensus.cc:2454] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:51.643371 6268 leader_election.cc:302] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:51.643529 7207 raft_consensus.cc:2781] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:51.643584 7207 raft_consensus.cc:482] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:51.643627 7207 raft_consensus.cc:3037] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.644443 7207 raft_consensus.cc:504] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.644737 7207 leader_election.cc:288] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:51.645107 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bf587ac1d8141e99fceab28d89e4ea9" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:51.645107 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bf587ac1d8141e99fceab28d89e4ea9" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:51.645236 6462 raft_consensus.cc:3037] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.645236 6593 raft_consensus.cc:3037] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.646139 6462 raft_consensus.cc:2454] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:51.646139 6593 raft_consensus.cc:2454] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:51.646430 6268 leader_election.cc:302] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:51.646575 7207 raft_consensus.cc:2781] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:51.646659 7207 raft_consensus.cc:686] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Becoming Leader. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Running, Role: LEADER
I20240501 07:59:51.646795 7207 consensus_queue.cc:237] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:51.648417 6056 catalog_manager.cc:5484] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a155eaa444d440d90fde420951458e4 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:51.690929 7329 raft_consensus.cc:482] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:51.691058 7329 raft_consensus.cc:504] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:51.691432 7329 leader_election.cc:288] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:51.691951 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "947b5d16224a42dcad709e7ddefc0b33" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:51.691974 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "947b5d16224a42dcad709e7ddefc0b33" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:51.692083 6593 raft_consensus.cc:2454] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:51.692152 6200 raft_consensus.cc:2454] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:51.692384 6399 leader_election.cc:302] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:51.692539 7329 raft_consensus.cc:2781] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:51.692600 7329 raft_consensus.cc:482] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:51.692662 7329 raft_consensus.cc:3037] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.693418 7329 raft_consensus.cc:504] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:51.693710 7329 leader_election.cc:288] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Requested vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:51.694072 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "947b5d16224a42dcad709e7ddefc0b33" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:51.694070 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "947b5d16224a42dcad709e7ddefc0b33" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:51.694176 6593 raft_consensus.cc:3037] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.694183 6200 raft_consensus.cc:3037] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.694906 6593 raft_consensus.cc:2454] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:51.694988 6200 raft_consensus.cc:2454] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:51.695206 6399 leader_election.cc:302] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:51.695428 7329 raft_consensus.cc:2781] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:51.695601 7329 raft_consensus.cc:686] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Becoming Leader. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Running, Role: LEADER
I20240501 07:59:51.695780 7329 consensus_queue.cc:237] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:51.697397 6056 catalog_manager.cc:5484] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:51.769140 7375 raft_consensus.cc:482] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:51.769304 7375 raft_consensus.cc:504] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:51.769779 7375 leader_election.cc:288] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 07:59:51.770630 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d929e6e09be943b68061ad4b1bb8e9f1" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:51.770795 6462 raft_consensus.cc:2454] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:51.771081 6137 leader_election.cc:302] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:51.771250 7375 raft_consensus.cc:2781] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:51.771318 7375 raft_consensus.cc:482] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:51.771353 7375 raft_consensus.cc:3037] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.772123 7375 raft_consensus.cc:504] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:51.772444 7375 leader_election.cc:288] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Requested vote from peers 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 07:59:51.772802 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d929e6e09be943b68061ad4b1bb8e9f1" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:51.772914 6462 raft_consensus.cc:3037] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.773754 6462 raft_consensus.cc:2454] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:51.774029 6137 leader_election.cc:302] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:51.774199 7375 raft_consensus.cc:2781] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:51.774310 7375 raft_consensus.cc:686] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Becoming Leader. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Running, Role: LEADER
I20240501 07:59:51.774402 6332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d929e6e09be943b68061ad4b1bb8e9f1" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:51.774428 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d929e6e09be943b68061ad4b1bb8e9f1" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:51.774456 7375 consensus_queue.cc:237] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:51.774538 6332 raft_consensus.cc:2454] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:51.776120 6056 catalog_manager.cc:5484] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:51.897114 7432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:51.897382 7432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:51.900506 7432 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240501 07:59:51.993968 6462 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:51.994299 6332 raft_consensus.cc:3037] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:51.995326 6332 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:51.995785 7375 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:51.996642 6593 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:51.997046 7207 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:51.998355 7375 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:52.000173 6462 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:52.000993 7207 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:52.005406 6594 raft_consensus.cc:1261] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:52.005918 7329 consensus_queue.cc:1035] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:52.009899 6200 raft_consensus.cc:1261] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:52.010351 7427 consensus_queue.cc:1035] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 110.677 ms
time per row: 0.0553384 ms
I20240501 07:59:52.060019 6053 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43232:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:52.065384 6442 tablet_service.cc:1467] Processing CreateTablet for tablet a97136a7017541eb8e4afb281c97e3cb (DEFAULT_TABLE table=rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.065595 6181 tablet_service.cc:1467] Processing CreateTablet for tablet a97136a7017541eb8e4afb281c97e3cb (DEFAULT_TABLE table=rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.065723 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97136a7017541eb8e4afb281c97e3cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.065853 6181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97136a7017541eb8e4afb281c97e3cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.066504 6572 tablet_service.cc:1467] Processing CreateTablet for tablet 16752f91ea894a7ba4017170c1dc6ec1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.066759 6572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16752f91ea894a7ba4017170c1dc6ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.067888 7363 tablet_bootstrap.cc:492] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:52.067891 7360 tablet_bootstrap.cc:492] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:52.068766 7363 tablet_bootstrap.cc:654] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.068794 7360 tablet_bootstrap.cc:654] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.069542 6573 tablet_service.cc:1467] Processing CreateTablet for tablet 5dae6c0b9c1d4009a93ed157e1f763c8 (DEFAULT_TABLE table=rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.069769 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dae6c0b9c1d4009a93ed157e1f763c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.069986 6181 tablet_service.cc:1467] Processing CreateTablet for tablet 5dae6c0b9c1d4009a93ed157e1f763c8 (DEFAULT_TABLE table=rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.070199 6181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dae6c0b9c1d4009a93ed157e1f763c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.070751 6442 tablet_service.cc:1467] Processing CreateTablet for tablet 16752f91ea894a7ba4017170c1dc6ec1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.070992 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16752f91ea894a7ba4017170c1dc6ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.072032 6180 tablet_service.cc:1467] Processing CreateTablet for tablet 16752f91ea894a7ba4017170c1dc6ec1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.072258 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16752f91ea894a7ba4017170c1dc6ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.073329 7480 tablet_bootstrap.cc:492] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:52.074064 7480 tablet_bootstrap.cc:654] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.074425 6312 tablet_service.cc:1467] Processing CreateTablet for tablet 5dae6c0b9c1d4009a93ed157e1f763c8 (DEFAULT_TABLE table=rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.074425 6311 tablet_service.cc:1467] Processing CreateTablet for tablet a97136a7017541eb8e4afb281c97e3cb (DEFAULT_TABLE table=rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.074705 6312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dae6c0b9c1d4009a93ed157e1f763c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.075011 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97136a7017541eb8e4afb281c97e3cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.076978 7484 tablet_bootstrap.cc:492] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:52.077144 7363 tablet_bootstrap.cc:492] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:52.077226 7363 ts_tablet_manager.cc:1367] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240501 07:59:52.077272 7360 tablet_bootstrap.cc:492] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:52.077361 7360 ts_tablet_manager.cc:1367] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 07:59:52.077636 7363 raft_consensus.cc:348] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.077737 7363 raft_consensus.cc:374] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.077769 7363 raft_consensus.cc:729] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.077834 7484 tablet_bootstrap.cc:654] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.077775 7360 raft_consensus.cc:348] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.077868 7360 raft_consensus.cc:374] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.077893 7360 raft_consensus.cc:729] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.077888 7363 consensus_queue.cc:260] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.077996 7360 consensus_queue.cc:260] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.078158 7363 ts_tablet_manager.cc:1398] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:52.078245 7360 ts_tablet_manager.cc:1398] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:52.078318 7363 tablet_bootstrap.cc:492] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:52.078397 7360 tablet_bootstrap.cc:492] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:52.078799 7484 tablet_bootstrap.cc:492] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:52.078878 7484 ts_tablet_manager.cc:1367] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:52.079144 7363 tablet_bootstrap.cc:654] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.079196 7360 tablet_bootstrap.cc:654] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.079284 7484 raft_consensus.cc:348] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.079375 7484 raft_consensus.cc:374] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.079402 7484 raft_consensus.cc:729] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.079524 7484 consensus_queue.cc:260] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.079782 7484 ts_tablet_manager.cc:1398] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:52.079970 7484 tablet_bootstrap.cc:492] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:52.080775 7484 tablet_bootstrap.cc:654] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.081838 7484 tablet_bootstrap.cc:492] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:52.081919 7484 ts_tablet_manager.cc:1367] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:52.082487 7360 tablet_bootstrap.cc:492] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:52.082446 7484 raft_consensus.cc:348] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.082554 7484 raft_consensus.cc:374] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.082561 7360 ts_tablet_manager.cc:1367] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 07:59:52.082588 7484 raft_consensus.cc:729] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.082701 7484 consensus_queue.cc:260] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.082980 7484 ts_tablet_manager.cc:1398] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:52.082955 7360 raft_consensus.cc:348] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.083048 7360 raft_consensus.cc:374] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.083079 7360 raft_consensus.cc:729] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.083184 7360 consensus_queue.cc:260] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.083421 7360 ts_tablet_manager.cc:1398] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:52.083557 7360 tablet_bootstrap.cc:492] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:52.084388 7360 tablet_bootstrap.cc:654] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.085440 7363 tablet_bootstrap.cc:492] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:52.085945 7360 tablet_bootstrap.cc:492] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:52.086014 7360 ts_tablet_manager.cc:1367] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:52.086369 7360 raft_consensus.cc:348] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.086447 7360 raft_consensus.cc:374] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.086472 7360 raft_consensus.cc:729] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.086555 7360 consensus_queue.cc:260] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.086721 7363 ts_tablet_manager.cc:1367] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240501 07:59:52.086769 7360 ts_tablet_manager.cc:1398] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:52.087119 7363 raft_consensus.cc:348] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.087203 7363 raft_consensus.cc:374] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.087224 7363 raft_consensus.cc:729] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.087330 7363 consensus_queue.cc:260] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.088063 7480 tablet_bootstrap.cc:492] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:52.088162 7363 ts_tablet_manager.cc:1398] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:52.088205 7480 ts_tablet_manager.cc:1367] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240501 07:59:52.088688 7480 raft_consensus.cc:348] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.088790 7480 raft_consensus.cc:374] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.088841 7480 raft_consensus.cc:729] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.088970 7480 consensus_queue.cc:260] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.089241 7480 ts_tablet_manager.cc:1398] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:52.089733 7480 tablet_bootstrap.cc:492] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:52.090536 7480 tablet_bootstrap.cc:654] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.091506 7480 tablet_bootstrap.cc:492] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:52.091585 7480 ts_tablet_manager.cc:1367] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:52.092003 7480 raft_consensus.cc:348] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.092109 7480 raft_consensus.cc:374] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.092151 7480 raft_consensus.cc:729] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.092267 7480 consensus_queue.cc:260] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.092494 7480 ts_tablet_manager.cc:1398] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:52.122006 7333 raft_consensus.cc:482] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:52.122117 7333 raft_consensus.cc:504] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.122469 7333 leader_election.cc:288] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:52.122856 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97136a7017541eb8e4afb281c97e3cb" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:52.122913 6332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97136a7017541eb8e4afb281c97e3cb" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:52.122994 6463 raft_consensus.cc:2454] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:52.123077 6332 raft_consensus.cc:2454] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:52.123276 6137 leader_election.cc:302] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:52.123445 7333 raft_consensus.cc:2781] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:52.123502 7333 raft_consensus.cc:482] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:52.123526 7333 raft_consensus.cc:3037] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.124366 7333 raft_consensus.cc:504] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.124660 7333 leader_election.cc:288] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:52.124990 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97136a7017541eb8e4afb281c97e3cb" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:52.125066 6332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97136a7017541eb8e4afb281c97e3cb" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:52.125119 6463 raft_consensus.cc:3037] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.125185 6332 raft_consensus.cc:3037] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.125873 6332 raft_consensus.cc:2454] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:52.125882 6463 raft_consensus.cc:2454] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:52.126124 6137 leader_election.cc:302] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:52.126286 7333 raft_consensus.cc:2781] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:52.126360 7333 raft_consensus.cc:686] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Becoming Leader. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Running, Role: LEADER
I20240501 07:59:52.126488 7333 consensus_queue.cc:237] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.128016 6050 catalog_manager.cc:5484] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:52.254120 7333 raft_consensus.cc:482] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:52.254269 7333 raft_consensus.cc:504] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.254724 7333 leader_election.cc:288] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:52.255227 6332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dae6c0b9c1d4009a93ed157e1f763c8" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:52.255249 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dae6c0b9c1d4009a93ed157e1f763c8" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:52.255386 6593 raft_consensus.cc:2454] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:52.255396 6332 raft_consensus.cc:2454] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:52.255673 6137 leader_election.cc:302] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:52.256017 7333 raft_consensus.cc:2781] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:52.256076 7333 raft_consensus.cc:482] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:52.256121 7333 raft_consensus.cc:3037] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.256922 7333 raft_consensus.cc:504] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.257256 7333 leader_election.cc:288] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Requested vote from peers 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:52.257627 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dae6c0b9c1d4009a93ed157e1f763c8" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:52.257665 6332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dae6c0b9c1d4009a93ed157e1f763c8" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:52.257740 6593 raft_consensus.cc:3037] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.257786 6332 raft_consensus.cc:3037] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.258561 6593 raft_consensus.cc:2454] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:52.258666 6332 raft_consensus.cc:2454] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:52.258850 6137 leader_election.cc:302] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 953b945e3e7e4c71a61aa99c1208ab1d, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:52.259004 7333 raft_consensus.cc:2781] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:52.259114 7333 raft_consensus.cc:686] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Becoming Leader. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Running, Role: LEADER
I20240501 07:59:52.259259 7333 consensus_queue.cc:237] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.260957 6050 catalog_manager.cc:5484] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:52.293406 7333 raft_consensus.cc:482] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:52.293550 7333 raft_consensus.cc:504] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.294077 7333 leader_election.cc:288] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:52.294554 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16752f91ea894a7ba4017170c1dc6ec1" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:52.294587 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16752f91ea894a7ba4017170c1dc6ec1" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:52.294685 6463 raft_consensus.cc:2454] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:52.294744 6593 raft_consensus.cc:2454] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:52.295049 6137 leader_election.cc:302] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:52.295238 7333 raft_consensus.cc:2781] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:52.295300 7333 raft_consensus.cc:482] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:52.295325 7333 raft_consensus.cc:3037] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.296185 7333 raft_consensus.cc:504] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.296500 7333 leader_election.cc:288] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:52.296900 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16752f91ea894a7ba4017170c1dc6ec1" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:52.296900 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16752f91ea894a7ba4017170c1dc6ec1" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:52.296998 6593 raft_consensus.cc:3037] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.297008 6463 raft_consensus.cc:3037] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.297660 6593 raft_consensus.cc:2454] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:52.297742 6463 raft_consensus.cc:2454] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:52.297962 6137 leader_election.cc:302] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 953b945e3e7e4c71a61aa99c1208ab1d, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:52.298116 7333 raft_consensus.cc:2781] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:52.298225 7333 raft_consensus.cc:686] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Becoming Leader. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Running, Role: LEADER
I20240501 07:59:52.298367 7333 consensus_queue.cc:237] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:52.300058 6050 catalog_manager.cc:5484] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } }
W20240501 07:59:52.425438 7494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:52.425695 7494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:52.428817 7494 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240501 07:59:52.510432 6463 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:52.511751 6332 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:52.512368 7333 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:52.512560 6593 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:52.512897 6463 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:52.513087 7375 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:52.513339 6332 raft_consensus.cc:1261] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:52.513351 7333 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:52.513373 7375 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:52.513908 6593 raft_consensus.cc:1261] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:52.514505 7333 consensus_queue.cc:1035] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:52.514773 7333 consensus_queue.cc:1035] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:52.525735 7334 mvcc.cc:205] Tried to move back new op lower bound from 7022798407717363712 to 7022798406855086080. Current Snapshot: MvccSnapshot[applied={T|T < 7022798407717363712}]
INSERT report
rows total: 2000
time total: 102.768 ms
time per row: 0.051384 ms
I20240501 07:59:52.574208 6050 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43232:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:52.579470 6573 tablet_service.cc:1467] Processing CreateTablet for tablet a4b3aaeeb72e47e08c938b47310dd402 (DEFAULT_TABLE table=rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.579470 6442 tablet_service.cc:1467] Processing CreateTablet for tablet a4b3aaeeb72e47e08c938b47310dd402 (DEFAULT_TABLE table=rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.579643 6312 tablet_service.cc:1467] Processing CreateTablet for tablet a4b3aaeeb72e47e08c938b47310dd402 (DEFAULT_TABLE table=rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.579766 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b3aaeeb72e47e08c938b47310dd402. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.579766 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b3aaeeb72e47e08c938b47310dd402. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.580010 6312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b3aaeeb72e47e08c938b47310dd402. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.580515 6311 tablet_service.cc:1467] Processing CreateTablet for tablet 41b9f1a284c34a9099cf1014db77f35c (DEFAULT_TABLE table=rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.580732 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b9f1a284c34a9099cf1014db77f35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.581897 7363 tablet_bootstrap.cc:492] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:52.581909 7480 tablet_bootstrap.cc:492] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:52.582378 7484 tablet_bootstrap.cc:492] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:52.582580 6310 tablet_service.cc:1467] Processing CreateTablet for tablet e952704c6a3d43aba833484bba9fddbf (DEFAULT_TABLE table=rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.582711 7363 tablet_bootstrap.cc:654] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.582712 7480 tablet_bootstrap.cc:654] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.582768 6310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e952704c6a3d43aba833484bba9fddbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.583246 7484 tablet_bootstrap.cc:654] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.584014 6180 tablet_service.cc:1467] Processing CreateTablet for tablet e952704c6a3d43aba833484bba9fddbf (DEFAULT_TABLE table=rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.584206 6442 tablet_service.cc:1467] Processing CreateTablet for tablet 41b9f1a284c34a9099cf1014db77f35c (DEFAULT_TABLE table=rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.584268 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e952704c6a3d43aba833484bba9fddbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.584442 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b9f1a284c34a9099cf1014db77f35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.584483 6573 tablet_service.cc:1467] Processing CreateTablet for tablet 41b9f1a284c34a9099cf1014db77f35c (DEFAULT_TABLE table=rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.584689 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b9f1a284c34a9099cf1014db77f35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.586154 6441 tablet_service.cc:1467] Processing CreateTablet for tablet e952704c6a3d43aba833484bba9fddbf (DEFAULT_TABLE table=rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:52.586360 6441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e952704c6a3d43aba833484bba9fddbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:52.587015 7363 tablet_bootstrap.cc:492] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:52.587091 7363 ts_tablet_manager.cc:1367] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240501 07:59:52.587175 7480 tablet_bootstrap.cc:492] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:52.587245 7480 ts_tablet_manager.cc:1367] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 07:59:52.587535 7363 raft_consensus.cc:348] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:52.587635 7363 raft_consensus.cc:374] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.587664 7363 raft_consensus.cc:729] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.587616 7480 raft_consensus.cc:348] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:52.587704 7480 raft_consensus.cc:374] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.587729 7480 raft_consensus.cc:729] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.587778 7363 consensus_queue.cc:260] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:52.587827 7360 tablet_bootstrap.cc:492] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:52.587853 7480 consensus_queue.cc:260] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:52.588066 7363 ts_tablet_manager.cc:1398] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:52.588101 7480 ts_tablet_manager.cc:1398] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:52.588205 7363 tablet_bootstrap.cc:492] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:52.588229 7480 tablet_bootstrap.cc:492] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:52.588624 7360 tablet_bootstrap.cc:654] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.589031 7480 tablet_bootstrap.cc:654] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.589031 7363 tablet_bootstrap.cc:654] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.590094 7360 tablet_bootstrap.cc:492] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:52.590185 7360 ts_tablet_manager.cc:1367] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:52.590590 7360 raft_consensus.cc:348] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:52.590695 7363 tablet_bootstrap.cc:492] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:52.590703 7360 raft_consensus.cc:374] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.590735 7360 raft_consensus.cc:729] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.590763 7363 ts_tablet_manager.cc:1367] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:52.590863 7360 consensus_queue.cc:260] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:52.591120 7360 ts_tablet_manager.cc:1398] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:52.591168 7363 raft_consensus.cc:348] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:52.591356 7363 raft_consensus.cc:374] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.591435 7363 raft_consensus.cc:729] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.592039 7484 tablet_bootstrap.cc:492] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:52.592131 7484 ts_tablet_manager.cc:1367] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 07:59:52.592212 7480 tablet_bootstrap.cc:492] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:52.592279 7480 ts_tablet_manager.cc:1367] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 07:59:52.592574 7484 raft_consensus.cc:348] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:52.592696 7484 raft_consensus.cc:374] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.592751 7484 raft_consensus.cc:729] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.592885 7484 consensus_queue.cc:260] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:52.593184 7484 ts_tablet_manager.cc:1398] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:52.593355 7484 tablet_bootstrap.cc:492] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:52.594069 7480 raft_consensus.cc:348] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:52.594166 7480 raft_consensus.cc:374] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.594194 7480 raft_consensus.cc:729] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.594240 7484 tablet_bootstrap.cc:654] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.594316 7480 consensus_queue.cc:260] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:52.594589 7480 ts_tablet_manager.cc:1398] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 07:59:52.595599 7484 tablet_bootstrap.cc:492] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:52.595677 7484 ts_tablet_manager.cc:1367] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:52.596138 7484 raft_consensus.cc:348] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:52.596243 7484 raft_consensus.cc:374] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.596282 7484 raft_consensus.cc:729] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.596397 7484 consensus_queue.cc:260] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:52.596653 7484 ts_tablet_manager.cc:1398] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:52.596771 7363 consensus_queue.cc:260] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:52.597041 7484 tablet_bootstrap.cc:492] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:52.597944 7484 tablet_bootstrap.cc:654] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.598621 7363 ts_tablet_manager.cc:1398] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240501 07:59:52.598824 7363 tablet_bootstrap.cc:492] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:52.599606 7484 tablet_bootstrap.cc:492] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:52.599632 7363 tablet_bootstrap.cc:654] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:52.599694 7484 ts_tablet_manager.cc:1367] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:52.600132 7484 raft_consensus.cc:348] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:52.600251 7484 raft_consensus.cc:374] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.600283 7484 raft_consensus.cc:729] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.600441 7484 consensus_queue.cc:260] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:52.600908 7363 tablet_bootstrap.cc:492] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:52.601119 7363 ts_tablet_manager.cc:1367] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:52.601120 7484 ts_tablet_manager.cc:1398] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:52.601775 7363 raft_consensus.cc:348] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:52.601871 7363 raft_consensus.cc:374] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:52.601898 7363 raft_consensus.cc:729] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:52.602025 7363 consensus_queue.cc:260] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:52.602252 7363 ts_tablet_manager.cc:1398] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:52.608660 7529 raft_consensus.cc:482] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:52.608758 7529 raft_consensus.cc:504] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:52.609141 7529 leader_election.cc:288] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:52.609511 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b3aaeeb72e47e08c938b47310dd402" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:52.609550 6594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b3aaeeb72e47e08c938b47310dd402" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:52.609649 6462 raft_consensus.cc:2454] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:52.609675 6594 raft_consensus.cc:2454] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:52.609925 6268 leader_election.cc:302] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:52.610196 7529 raft_consensus.cc:2781] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:52.610260 7529 raft_consensus.cc:482] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:52.610294 7529 raft_consensus.cc:3037] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.611096 7529 raft_consensus.cc:504] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:52.611387 7529 leader_election.cc:288] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:52.611726 6594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b3aaeeb72e47e08c938b47310dd402" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:52.611724 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b3aaeeb72e47e08c938b47310dd402" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:52.611830 6594 raft_consensus.cc:3037] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.611846 6462 raft_consensus.cc:3037] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.612565 6594 raft_consensus.cc:2454] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:52.612576 6462 raft_consensus.cc:2454] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:52.612830 6268 leader_election.cc:302] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:52.612985 7529 raft_consensus.cc:2781] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:52.613065 7529 raft_consensus.cc:686] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Becoming Leader. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Running, Role: LEADER
I20240501 07:59:52.613207 7529 consensus_queue.cc:237] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:52.614872 6054 catalog_manager.cc:5484] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a155eaa444d440d90fde420951458e4 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:52.630204 7375 raft_consensus.cc:482] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:52.630360 7375 raft_consensus.cc:504] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:52.630766 7375 leader_election.cc:288] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 07:59:52.631181 6332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e952704c6a3d43aba833484bba9fddbf" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:52.631213 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e952704c6a3d43aba833484bba9fddbf" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:52.631318 6332 raft_consensus.cc:2454] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:52.631345 6462 raft_consensus.cc:2454] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:52.631587 6137 leader_election.cc:302] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:52.631791 7375 raft_consensus.cc:2781] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:52.631847 7375 raft_consensus.cc:482] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:52.631875 7375 raft_consensus.cc:3037] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.632683 7375 raft_consensus.cc:504] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:52.633020 7375 leader_election.cc:288] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Requested vote from peers 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 07:59:52.633325 6332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e952704c6a3d43aba833484bba9fddbf" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:52.633430 6332 raft_consensus.cc:3037] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.633448 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e952704c6a3d43aba833484bba9fddbf" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:52.633575 6462 raft_consensus.cc:3037] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.634312 6462 raft_consensus.cc:2454] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:52.634582 6137 leader_election.cc:302] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:52.634815 7375 raft_consensus.cc:2781] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:52.634910 7375 raft_consensus.cc:686] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Becoming Leader. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Running, Role: LEADER
I20240501 07:59:52.635048 7375 consensus_queue.cc:237] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:52.636715 6054 catalog_manager.cc:5484] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:52.648705 6332 raft_consensus.cc:2454] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:52.782227 7529 raft_consensus.cc:482] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:52.782354 7529 raft_consensus.cc:504] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:52.782730 7529 leader_election.cc:288] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:52.783172 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b9f1a284c34a9099cf1014db77f35c" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:52.783260 6594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b9f1a284c34a9099cf1014db77f35c" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:52.783324 6462 raft_consensus.cc:2454] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:52.783422 6594 raft_consensus.cc:2454] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:52.783617 6268 leader_election.cc:302] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:52.783794 7529 raft_consensus.cc:2781] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:52.783856 7529 raft_consensus.cc:482] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:52.783885 7529 raft_consensus.cc:3037] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.784816 7529 raft_consensus.cc:504] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:52.785125 7529 leader_election.cc:288] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:52.785503 6594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b9f1a284c34a9099cf1014db77f35c" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:52.785552 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b9f1a284c34a9099cf1014db77f35c" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:52.785609 6594 raft_consensus.cc:3037] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.785697 6462 raft_consensus.cc:3037] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:52.786334 6594 raft_consensus.cc:2454] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:52.786609 6268 leader_election.cc:302] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:52.786770 7529 raft_consensus.cc:2781] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:52.786752 6462 raft_consensus.cc:2454] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:52.786916 7529 raft_consensus.cc:686] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Becoming Leader. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Running, Role: LEADER
I20240501 07:59:52.787088 7529 consensus_queue.cc:237] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:52.788846 6054 catalog_manager.cc:5484] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a155eaa444d440d90fde420951458e4 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:52.934352 7547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:52.934638 7547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:52.938025 7547 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240501 07:59:53.020198 6594 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:53.020411 6331 raft_consensus.cc:1261] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:53.020509 6462 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:53.020545 6593 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:53.020798 6463 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:53.020812 6461 raft_consensus.cc:1261] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:53.021174 7375 consensus_queue.cc:1035] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:53.021229 7207 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:53.021318 7333 consensus_queue.cc:1035] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:53.021436 7190 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:53.021570 7207 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:53.021714 7190 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:53.036238 7472 mvcc.cc:205] Tried to move back new op lower bound from 7022798409804931072 to 7022798408234418176. Current Snapshot: MvccSnapshot[applied={T|T < 7022798409804931072}]
I20240501 07:59:53.041285 7326 mvcc.cc:205] Tried to move back new op lower bound from 7022798409805811712 to 7022798408144924672. Current Snapshot: MvccSnapshot[applied={T|T < 7022798409793007616}]
I20240501 07:59:53.067160 7331 mvcc.cc:205] Tried to move back new op lower bound from 7022798409805680640 to 7022798408857219072. Current Snapshot: MvccSnapshot[applied={T|T < 7022798409805680640 or (T in {7022798409805680640})}]
INSERT report
rows total: 2000
time total: 97.1763 ms
time per row: 0.0485882 ms
I20240501 07:59:53.082136 6056 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43232:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:53.087548 6573 tablet_service.cc:1467] Processing CreateTablet for tablet 5a31c7d3de834d1aab1ffa16e6df7e15 (DEFAULT_TABLE table=rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.087595 6180 tablet_service.cc:1467] Processing CreateTablet for tablet 5a31c7d3de834d1aab1ffa16e6df7e15 (DEFAULT_TABLE table=rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.087878 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a31c7d3de834d1aab1ffa16e6df7e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.087908 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a31c7d3de834d1aab1ffa16e6df7e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.089637 7360 tablet_bootstrap.cc:492] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:53.090333 6572 tablet_service.cc:1467] Processing CreateTablet for tablet 5c411a11ec484cd5bd063e5fc3868e29 (DEFAULT_TABLE table=rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.090535 7360 tablet_bootstrap.cc:654] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.090608 6572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c411a11ec484cd5bd063e5fc3868e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.090705 6310 tablet_service.cc:1467] Processing CreateTablet for tablet 5a31c7d3de834d1aab1ffa16e6df7e15 (DEFAULT_TABLE table=rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.090981 6310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a31c7d3de834d1aab1ffa16e6df7e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.091082 6574 tablet_service.cc:1467] Processing CreateTablet for tablet 6b552a44c67940819f9ef64c68585eb1 (DEFAULT_TABLE table=rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.091305 6574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b552a44c67940819f9ef64c68585eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.092876 7484 tablet_bootstrap.cc:492] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:53.093094 6311 tablet_service.cc:1467] Processing CreateTablet for tablet 6b552a44c67940819f9ef64c68585eb1 (DEFAULT_TABLE table=rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.093291 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b552a44c67940819f9ef64c68585eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.093691 7484 tablet_bootstrap.cc:654] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.094744 7484 tablet_bootstrap.cc:492] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:53.094830 7484 ts_tablet_manager.cc:1367] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:53.095252 7484 raft_consensus.cc:348] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.095346 7484 raft_consensus.cc:374] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.095374 7484 raft_consensus.cc:729] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.095410 6441 tablet_service.cc:1467] Processing CreateTablet for tablet 5c411a11ec484cd5bd063e5fc3868e29 (DEFAULT_TABLE table=rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.095490 7484 consensus_queue.cc:260] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.095659 6441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c411a11ec484cd5bd063e5fc3868e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.095762 7484 ts_tablet_manager.cc:1398] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:53.095901 7484 tablet_bootstrap.cc:492] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:53.096724 6181 tablet_service.cc:1467] Processing CreateTablet for tablet 6b552a44c67940819f9ef64c68585eb1 (DEFAULT_TABLE table=rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.096968 6181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b552a44c67940819f9ef64c68585eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.097602 7484 tablet_bootstrap.cc:654] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.098695 7363 tablet_bootstrap.cc:492] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:53.099478 7363 tablet_bootstrap.cc:654] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.099711 6180 tablet_service.cc:1467] Processing CreateTablet for tablet 5c411a11ec484cd5bd063e5fc3868e29 (DEFAULT_TABLE table=rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.099980 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c411a11ec484cd5bd063e5fc3868e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.100859 7363 tablet_bootstrap.cc:492] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:53.100941 7363 ts_tablet_manager.cc:1367] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:53.101318 7363 raft_consensus.cc:348] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:53.101408 7363 raft_consensus.cc:374] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.101428 7363 raft_consensus.cc:729] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.101531 7363 consensus_queue.cc:260] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:53.101763 7363 ts_tablet_manager.cc:1398] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:53.102437 7360 tablet_bootstrap.cc:492] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:53.102517 7360 ts_tablet_manager.cc:1367] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240501 07:59:53.102594 7484 tablet_bootstrap.cc:492] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:53.102671 7484 ts_tablet_manager.cc:1367] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240501 07:59:53.102936 7360 raft_consensus.cc:348] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.103037 7360 raft_consensus.cc:374] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.103067 7360 raft_consensus.cc:729] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.103116 7484 raft_consensus.cc:348] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.103216 7484 raft_consensus.cc:374] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.103183 7360 consensus_queue.cc:260] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.103248 7484 raft_consensus.cc:729] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.103360 7484 consensus_queue.cc:260] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.103462 7360 ts_tablet_manager.cc:1398] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:53.103616 7360 tablet_bootstrap.cc:492] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:53.103647 7484 ts_tablet_manager.cc:1398] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:53.104476 7360 tablet_bootstrap.cc:654] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.105494 7360 tablet_bootstrap.cc:492] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:53.105569 7360 ts_tablet_manager.cc:1367] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:53.105914 7360 raft_consensus.cc:348] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.106021 7360 raft_consensus.cc:374] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.106057 7360 raft_consensus.cc:729] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.106163 7360 consensus_queue.cc:260] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.106398 7360 ts_tablet_manager.cc:1398] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:53.106547 7360 tablet_bootstrap.cc:492] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:53.107268 7360 tablet_bootstrap.cc:654] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.109604 7597 tablet_bootstrap.cc:492] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:53.110198 7360 tablet_bootstrap.cc:492] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:53.110285 7360 ts_tablet_manager.cc:1367] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 07:59:53.110460 7597 tablet_bootstrap.cc:654] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.110731 7360 raft_consensus.cc:348] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:53.110827 7360 raft_consensus.cc:374] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.110857 7360 raft_consensus.cc:729] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.110973 7360 consensus_queue.cc:260] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:53.111222 7360 ts_tablet_manager.cc:1398] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:53.111483 7597 tablet_bootstrap.cc:492] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:53.111570 7597 ts_tablet_manager.cc:1367] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:53.112051 7597 raft_consensus.cc:348] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.112165 7597 raft_consensus.cc:374] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.112203 7597 raft_consensus.cc:729] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.112334 7597 consensus_queue.cc:260] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.112600 7597 ts_tablet_manager.cc:1398] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:53.112777 7597 tablet_bootstrap.cc:492] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:53.113998 7597 tablet_bootstrap.cc:654] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.115309 7597 tablet_bootstrap.cc:492] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:53.115383 7597 ts_tablet_manager.cc:1367] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:53.115736 7597 raft_consensus.cc:348] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:53.115825 7597 raft_consensus.cc:374] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.115854 7597 raft_consensus.cc:729] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.116031 7597 consensus_queue.cc:260] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:53.116277 7597 ts_tablet_manager.cc:1398] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:53.116497 7597 tablet_bootstrap.cc:492] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:53.117251 7597 tablet_bootstrap.cc:654] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.118168 7597 tablet_bootstrap.cc:492] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:53.118247 7597 ts_tablet_manager.cc:1367] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:53.118692 7597 raft_consensus.cc:348] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.118806 7597 raft_consensus.cc:374] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.118839 7597 raft_consensus.cc:729] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.118968 7597 consensus_queue.cc:260] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.119253 7597 ts_tablet_manager.cc:1398] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:53.167039 7577 raft_consensus.cc:482] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:53.167210 7577 raft_consensus.cc:504] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.167641 7577 leader_election.cc:288] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:53.168114 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a31c7d3de834d1aab1ffa16e6df7e15" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:53.168186 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a31c7d3de834d1aab1ffa16e6df7e15" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:53.168255 6593 raft_consensus.cc:2454] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:53.168323 6200 raft_consensus.cc:2454] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:53.168569 6268 leader_election.cc:302] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:53.168746 7577 raft_consensus.cc:2781] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:53.168802 7577 raft_consensus.cc:482] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:53.168846 7577 raft_consensus.cc:3037] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.169823 7577 raft_consensus.cc:504] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.170153 7577 leader_election.cc:288] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Requested vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:53.170500 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a31c7d3de834d1aab1ffa16e6df7e15" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:53.170609 6593 raft_consensus.cc:3037] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.170580 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a31c7d3de834d1aab1ffa16e6df7e15" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:53.170701 6200 raft_consensus.cc:3037] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.171345 6593 raft_consensus.cc:2454] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:53.171491 6200 raft_consensus.cc:2454] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:53.171638 6268 leader_election.cc:302] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:53.171792 7577 raft_consensus.cc:2781] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:53.171871 7577 raft_consensus.cc:686] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Becoming Leader. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Running, Role: LEADER
I20240501 07:59:53.172058 7577 consensus_queue.cc:237] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.174343 6054 catalog_manager.cc:5484] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a155eaa444d440d90fde420951458e4 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:53.184902 7522 raft_consensus.cc:482] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:53.185039 7522 raft_consensus.cc:504] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.185503 7522 leader_election.cc:288] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:53.185923 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b552a44c67940819f9ef64c68585eb1" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:53.185940 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b552a44c67940819f9ef64c68585eb1" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:53.186069 6593 raft_consensus.cc:2454] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:53.186070 6331 raft_consensus.cc:2454] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 0.
I20240501 07:59:53.186350 6137 leader_election.cc:302] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 953b945e3e7e4c71a61aa99c1208ab1d, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:53.186522 7522 raft_consensus.cc:2781] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:53.186578 7522 raft_consensus.cc:482] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:53.186606 7522 raft_consensus.cc:3037] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.187462 7522 raft_consensus.cc:504] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.187772 7522 leader_election.cc:288] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Requested vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:53.188104 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b552a44c67940819f9ef64c68585eb1" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:53.188166 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b552a44c67940819f9ef64c68585eb1" candidate_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:53.188217 6593 raft_consensus.cc:3037] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.188277 6331 raft_consensus.cc:3037] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.189046 6331 raft_consensus.cc:2454] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:53.189110 6593 raft_consensus.cc:2454] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 953b945e3e7e4c71a61aa99c1208ab1d in term 1.
I20240501 07:59:53.189460 6137 leader_election.cc:302] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 953b945e3e7e4c71a61aa99c1208ab1d, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:53.190586 7522 raft_consensus.cc:2781] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:53.190687 7522 raft_consensus.cc:686] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Becoming Leader. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Running, Role: LEADER
I20240501 07:59:53.190825 7522 consensus_queue.cc:237] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.192596 6054 catalog_manager.cc:5484] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:53.208340 7588 raft_consensus.cc:482] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:53.208470 7588 raft_consensus.cc:504] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:53.208925 7588 leader_election.cc:288] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:53.209414 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c411a11ec484cd5bd063e5fc3868e29" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:53.209445 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c411a11ec484cd5bd063e5fc3868e29" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:53.209571 6593 raft_consensus.cc:2454] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:53.209571 6200 raft_consensus.cc:2454] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:53.209851 6399 leader_election.cc:302] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:53.210044 7588 raft_consensus.cc:2781] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:53.210105 7588 raft_consensus.cc:482] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:53.210143 7588 raft_consensus.cc:3037] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.211251 7588 raft_consensus.cc:504] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:53.211592 7588 leader_election.cc:288] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Requested vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:53.212078 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c411a11ec484cd5bd063e5fc3868e29" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:53.212085 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c411a11ec484cd5bd063e5fc3868e29" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:53.212214 6200 raft_consensus.cc:3037] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.212214 6593 raft_consensus.cc:3037] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.213191 6593 raft_consensus.cc:2454] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:53.213191 6200 raft_consensus.cc:2454] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:53.213527 6399 leader_election.cc:302] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:53.213706 7588 raft_consensus.cc:2781] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:53.213804 7588 raft_consensus.cc:686] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Becoming Leader. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Running, Role: LEADER
I20240501 07:59:53.213953 7588 consensus_queue.cc:237] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:53.215689 6054 catalog_manager.cc:5484] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } }
W20240501 07:59:53.338184 7604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:53.338479 7604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:53.341652 7604 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240501 07:59:53.422554 6200 raft_consensus.cc:1261] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:53.422667 6593 raft_consensus.cc:1261] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:53.423027 7529 consensus_queue.cc:1035] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:53.428244 6331 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:53.428684 7333 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:53.429677 7529 consensus_queue.cc:1035] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:53.430197 6593 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:53.430613 7333 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:53.432884 6594 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:53.433296 7467 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:53.435557 6200 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:53.437937 7467 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 120.908 ms
time per row: 0.0604538 ms
I20240501 07:59:53.505038 6056 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43232:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:53.511521 6442 tablet_service.cc:1467] Processing CreateTablet for tablet 7a9dcf4944c34707a8cd0acebc8f9029 (DEFAULT_TABLE table=rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.511857 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a9dcf4944c34707a8cd0acebc8f9029. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.511881 6573 tablet_service.cc:1467] Processing CreateTablet for tablet 7a9dcf4944c34707a8cd0acebc8f9029 (DEFAULT_TABLE table=rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.511909 6441 tablet_service.cc:1467] Processing CreateTablet for tablet 4e60dad8ddd2402faae35be18662c8c3 (DEFAULT_TABLE table=rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.512176 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a9dcf4944c34707a8cd0acebc8f9029. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.512151 6180 tablet_service.cc:1467] Processing CreateTablet for tablet 4e60dad8ddd2402faae35be18662c8c3 (DEFAULT_TABLE table=rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.512413 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e60dad8ddd2402faae35be18662c8c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.512445 6311 tablet_service.cc:1467] Processing CreateTablet for tablet 4e60dad8ddd2402faae35be18662c8c3 (DEFAULT_TABLE table=rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.512681 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e60dad8ddd2402faae35be18662c8c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.514289 7597 tablet_bootstrap.cc:492] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:53.514698 7363 tablet_bootstrap.cc:492] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:53.514766 7484 tablet_bootstrap.cc:492] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:53.514796 6442 tablet_service.cc:1467] Processing CreateTablet for tablet 33863ce53ec14aab9040f75e0360e0af (DEFAULT_TABLE table=rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.515022 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33863ce53ec14aab9040f75e0360e0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.515188 7597 tablet_bootstrap.cc:654] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.515507 7484 tablet_bootstrap.cc:654] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.515509 7363 tablet_bootstrap.cc:654] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.516093 7360 tablet_bootstrap.cc:492] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:53.516237 6441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e60dad8ddd2402faae35be18662c8c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.516841 7484 tablet_bootstrap.cc:492] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:53.516868 7360 tablet_bootstrap.cc:654] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.516922 7484 ts_tablet_manager.cc:1367] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 07:59:53.517371 7484 raft_consensus.cc:348] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.517484 7484 raft_consensus.cc:374] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.517516 7484 raft_consensus.cc:729] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.517634 7484 consensus_queue.cc:260] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.517905 7484 ts_tablet_manager.cc:1398] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 07:59:53.518308 6311 tablet_service.cc:1467] Processing CreateTablet for tablet 33863ce53ec14aab9040f75e0360e0af (DEFAULT_TABLE table=rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.518308 6573 tablet_service.cc:1467] Processing CreateTablet for tablet 33863ce53ec14aab9040f75e0360e0af (DEFAULT_TABLE table=rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.518525 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33863ce53ec14aab9040f75e0360e0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.518530 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33863ce53ec14aab9040f75e0360e0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.520529 7484 tablet_bootstrap.cc:492] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:53.521129 6180 tablet_service.cc:1467] Processing CreateTablet for tablet 7a9dcf4944c34707a8cd0acebc8f9029 (DEFAULT_TABLE table=rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:53.521260 7484 tablet_bootstrap.cc:654] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.521425 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a9dcf4944c34707a8cd0acebc8f9029. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:53.522271 7484 tablet_bootstrap.cc:492] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:53.522353 7484 ts_tablet_manager.cc:1367] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:53.522747 7363 tablet_bootstrap.cc:492] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:53.522833 7363 ts_tablet_manager.cc:1367] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240501 07:59:53.522805 7484 raft_consensus.cc:348] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.522956 7484 raft_consensus.cc:374] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.522995 7484 raft_consensus.cc:729] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.523116 7484 consensus_queue.cc:260] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.523274 7363 raft_consensus.cc:348] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:53.523371 7363 raft_consensus.cc:374] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.523394 7484 ts_tablet_manager.cc:1398] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:53.523404 7363 raft_consensus.cc:729] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.523519 7363 consensus_queue.cc:260] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:53.523761 7363 ts_tablet_manager.cc:1398] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:53.523952 7363 tablet_bootstrap.cc:492] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:53.524843 7363 tablet_bootstrap.cc:654] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.525038 7360 tablet_bootstrap.cc:492] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:53.525112 7360 ts_tablet_manager.cc:1367] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240501 07:59:53.525269 7597 tablet_bootstrap.cc:492] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:53.525339 7597 ts_tablet_manager.cc:1367] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240501 07:59:53.525540 7360 raft_consensus.cc:348] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.525633 7360 raft_consensus.cc:374] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.525662 7360 raft_consensus.cc:729] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.525779 7360 consensus_queue.cc:260] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.525770 7597 raft_consensus.cc:348] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:53.525914 7597 raft_consensus.cc:374] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.525943 7597 raft_consensus.cc:729] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.525954 7363 tablet_bootstrap.cc:492] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:53.526031 7363 ts_tablet_manager.cc:1367] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:53.526034 7360 ts_tablet_manager.cc:1398] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:53.526046 7597 consensus_queue.cc:260] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:53.526196 7360 tablet_bootstrap.cc:492] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:53.526296 7597 ts_tablet_manager.cc:1398] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:53.526460 7597 tablet_bootstrap.cc:492] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:53.527036 7360 tablet_bootstrap.cc:654] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.527117 7363 raft_consensus.cc:348] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.527215 7363 raft_consensus.cc:374] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.527241 7363 raft_consensus.cc:729] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.527380 7363 consensus_queue.cc:260] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.527446 7597 tablet_bootstrap.cc:654] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.527706 7363 ts_tablet_manager.cc:1398] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:53.527858 7363 tablet_bootstrap.cc:492] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:53.528867 7360 tablet_bootstrap.cc:492] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:53.528966 7360 ts_tablet_manager.cc:1367] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:53.529026 7363 tablet_bootstrap.cc:654] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:53.529426 7360 raft_consensus.cc:348] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:53.529539 7360 raft_consensus.cc:374] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.529574 7360 raft_consensus.cc:729] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.529686 7360 consensus_queue.cc:260] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:53.529953 7360 ts_tablet_manager.cc:1398] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:53.530225 7363 tablet_bootstrap.cc:492] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:53.530349 7363 ts_tablet_manager.cc:1367] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:53.530961 7363 raft_consensus.cc:348] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.531069 7363 raft_consensus.cc:374] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.531100 7363 raft_consensus.cc:729] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.531194 7597 tablet_bootstrap.cc:492] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:53.531283 7597 ts_tablet_manager.cc:1367] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240501 07:59:53.531339 7363 consensus_queue.cc:260] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.531646 7363 ts_tablet_manager.cc:1398] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:53.531740 7597 raft_consensus.cc:348] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.531849 7597 raft_consensus.cc:374] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:53.531888 7597 raft_consensus.cc:729] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:53.532068 7597 consensus_queue.cc:260] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.532333 7597 ts_tablet_manager.cc:1398] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:53.573508 7640 raft_consensus.cc:482] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:53.573652 7640 raft_consensus.cc:504] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.574088 7640 leader_election.cc:288] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:53.574555 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33863ce53ec14aab9040f75e0360e0af" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:53.574551 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33863ce53ec14aab9040f75e0360e0af" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:53.574705 6331 raft_consensus.cc:2454] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:53.574703 6592 raft_consensus.cc:2454] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:53.575029 6399 leader_election.cc:302] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:53.575194 7640 raft_consensus.cc:2781] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:53.575253 7640 raft_consensus.cc:482] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:53.575289 7640 raft_consensus.cc:3037] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.576324 7640 raft_consensus.cc:504] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.576680 7640 leader_election.cc:288] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Requested vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:53.577014 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33863ce53ec14aab9040f75e0360e0af" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:53.577061 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33863ce53ec14aab9040f75e0360e0af" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:53.577118 6592 raft_consensus.cc:3037] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.577183 6331 raft_consensus.cc:3037] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.577915 6592 raft_consensus.cc:2454] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:53.577958 6331 raft_consensus.cc:2454] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:53.578172 6399 leader_election.cc:302] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:53.578465 7640 raft_consensus.cc:2781] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:53.578547 7640 raft_consensus.cc:686] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Becoming Leader. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Running, Role: LEADER
I20240501 07:59:53.578667 7640 consensus_queue.cc:237] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.580368 6054 catalog_manager.cc:5484] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:53.616768 7640 raft_consensus.cc:482] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:53.616911 7640 raft_consensus.cc:504] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.617316 7640 leader_election.cc:288] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:53.617655 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e60dad8ddd2402faae35be18662c8c3" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:53.617723 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e60dad8ddd2402faae35be18662c8c3" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:53.617786 6200 raft_consensus.cc:2454] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:53.617867 6331 raft_consensus.cc:2454] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:53.618108 6399 leader_election.cc:302] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:53.618259 7640 raft_consensus.cc:2781] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:53.618315 7640 raft_consensus.cc:482] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:53.618340 7640 raft_consensus.cc:3037] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.619130 7640 raft_consensus.cc:504] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.619441 7640 leader_election.cc:288] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Requested vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:53.619788 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e60dad8ddd2402faae35be18662c8c3" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:53.619940 6331 raft_consensus.cc:3037] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.619967 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e60dad8ddd2402faae35be18662c8c3" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:53.620105 6200 raft_consensus.cc:3037] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.620836 6331 raft_consensus.cc:2454] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:53.621006 6200 raft_consensus.cc:2454] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:53.621321 6399 leader_election.cc:302] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:53.621521 7640 raft_consensus.cc:2781] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:53.621616 7640 raft_consensus.cc:686] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Becoming Leader. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Running, Role: LEADER
I20240501 07:59:53.621755 7640 consensus_queue.cc:237] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:53.623330 6054 catalog_manager.cc:5484] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:53.811686 7581 raft_consensus.cc:482] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:53.811838 7581 raft_consensus.cc:504] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:53.812345 7581 leader_election.cc:288] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:53.812788 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a9dcf4944c34707a8cd0acebc8f9029" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:53.812856 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a9dcf4944c34707a8cd0acebc8f9029" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:53.812969 6462 raft_consensus.cc:2454] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 0.
I20240501 07:59:53.812989 6200 raft_consensus.cc:2454] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 0.
I20240501 07:59:53.813287 6530 leader_election.cc:302] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 953b945e3e7e4c71a61aa99c1208ab1d, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:53.813472 7581 raft_consensus.cc:2781] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:53.813545 7581 raft_consensus.cc:482] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:53.813583 7581 raft_consensus.cc:3037] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.814499 7581 raft_consensus.cc:504] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:53.814870 7581 leader_election.cc:288] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:53.815176 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a9dcf4944c34707a8cd0acebc8f9029" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:53.815237 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a9dcf4944c34707a8cd0acebc8f9029" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:53.815289 6462 raft_consensus.cc:3037] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.815361 6200 raft_consensus.cc:3037] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:53.816076 6462 raft_consensus.cc:2454] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 1.
I20240501 07:59:53.816097 6200 raft_consensus.cc:2454] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 1.
I20240501 07:59:53.816350 6530 leader_election.cc:302] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 953b945e3e7e4c71a61aa99c1208ab1d, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:53.816514 7581 raft_consensus.cc:2781] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:53.816684 7581 raft_consensus.cc:686] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Becoming Leader. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Running, Role: LEADER
I20240501 07:59:53.816823 7581 consensus_queue.cc:237] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } }
I20240501 07:59:53.818444 6054 catalog_manager.cc:5484] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:53.999676 7655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:53.999955 7655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:54.003322 7655 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 07:59:54.015377 7640 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.018656 7640 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_7'
I20240501 07:59:54.087848 6462 raft_consensus.cc:1261] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.087914 6200 raft_consensus.cc:1261] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.088930 7653 consensus_queue.cc:1035] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.092999 7581 consensus_queue.cc:1035] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.108688 6331 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.109400 6201 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.109898 7588 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.110224 7588 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 90.9426 ms
time per row: 0.0454713 ms
I20240501 07:59:54.149991 6054 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43232:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:54.155462 6180 tablet_service.cc:1467] Processing CreateTablet for tablet 98d43e2703b14f9ebf926b5f2e2b2c56 (DEFAULT_TABLE table=rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.155798 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98d43e2703b14f9ebf926b5f2e2b2c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.155752 6311 tablet_service.cc:1467] Processing CreateTablet for tablet 98d43e2703b14f9ebf926b5f2e2b2c56 (DEFAULT_TABLE table=rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.156062 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98d43e2703b14f9ebf926b5f2e2b2c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.156384 6442 tablet_service.cc:1467] Processing CreateTablet for tablet 98d43e2703b14f9ebf926b5f2e2b2c56 (DEFAULT_TABLE table=rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.156625 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98d43e2703b14f9ebf926b5f2e2b2c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.157613 7711 tablet_bootstrap.cc:492] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:54.158375 7711 tablet_bootstrap.cc:654] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.158650 7713 tablet_bootstrap.cc:492] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:54.159209 6311 tablet_service.cc:1467] Processing CreateTablet for tablet 96386dd2c62e45859e1f3b30510e8969 (DEFAULT_TABLE table=rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.159454 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96386dd2c62e45859e1f3b30510e8969. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.159595 7713 tablet_bootstrap.cc:654] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.159627 6180 tablet_service.cc:1467] Processing CreateTablet for tablet 96386dd2c62e45859e1f3b30510e8969 (DEFAULT_TABLE table=rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.159847 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96386dd2c62e45859e1f3b30510e8969. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.160343 6573 tablet_service.cc:1467] Processing CreateTablet for tablet 96386dd2c62e45859e1f3b30510e8969 (DEFAULT_TABLE table=rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.160596 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96386dd2c62e45859e1f3b30510e8969. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.159633 6181 tablet_service.cc:1467] Processing CreateTablet for tablet dbc4b418e4c5493fa8a7734740225200 (DEFAULT_TABLE table=rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.161972 6181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbc4b418e4c5493fa8a7734740225200. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.162364 7716 tablet_bootstrap.cc:492] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:54.162483 6572 tablet_service.cc:1467] Processing CreateTablet for tablet dbc4b418e4c5493fa8a7734740225200 (DEFAULT_TABLE table=rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.162660 6572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbc4b418e4c5493fa8a7734740225200. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.163236 7716 tablet_bootstrap.cc:654] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.164577 6442 tablet_service.cc:1467] Processing CreateTablet for tablet dbc4b418e4c5493fa8a7734740225200 (DEFAULT_TABLE table=rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.164637 7713 tablet_bootstrap.cc:492] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:54.164760 7713 ts_tablet_manager.cc:1367] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20240501 07:59:54.164844 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbc4b418e4c5493fa8a7734740225200. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.165200 7713 raft_consensus.cc:348] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.165297 7713 raft_consensus.cc:374] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.165318 7713 raft_consensus.cc:729] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.165450 7713 consensus_queue.cc:260] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.165719 7713 ts_tablet_manager.cc:1398] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:54.166656 7712 tablet_bootstrap.cc:492] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:54.167225 7713 tablet_bootstrap.cc:492] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:54.167454 7712 tablet_bootstrap.cc:654] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.168133 7713 tablet_bootstrap.cc:654] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.168592 7712 tablet_bootstrap.cc:492] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:54.168726 7712 ts_tablet_manager.cc:1367] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:54.168742 7711 tablet_bootstrap.cc:492] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:54.168826 7711 ts_tablet_manager.cc:1367] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240501 07:59:54.169241 7711 raft_consensus.cc:348] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.169374 7711 raft_consensus.cc:374] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.169315 7712 raft_consensus.cc:348] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.169405 7711 raft_consensus.cc:729] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.169411 7712 raft_consensus.cc:374] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.169448 7712 raft_consensus.cc:729] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.169536 7711 consensus_queue.cc:260] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.169560 7712 consensus_queue.cc:260] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.169744 7713 tablet_bootstrap.cc:492] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:54.169821 7711 ts_tablet_manager.cc:1398] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:54.169829 7713 ts_tablet_manager.cc:1367] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:54.169999 7711 tablet_bootstrap.cc:492] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:54.170265 7713 raft_consensus.cc:348] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.170338 7712 ts_tablet_manager.cc:1398] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:54.170361 7713 raft_consensus.cc:374] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.170392 7713 raft_consensus.cc:729] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.170497 7712 tablet_bootstrap.cc:492] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:54.170523 7713 consensus_queue.cc:260] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.170856 7711 tablet_bootstrap.cc:654] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.170897 7713 ts_tablet_manager.cc:1398] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:54.171734 7712 tablet_bootstrap.cc:654] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.171877 7716 tablet_bootstrap.cc:492] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:54.171995 7716 ts_tablet_manager.cc:1367] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 07:59:54.172437 7716 raft_consensus.cc:348] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.172549 7716 raft_consensus.cc:374] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.172581 7716 raft_consensus.cc:729] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.172691 7716 consensus_queue.cc:260] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.172976 7716 ts_tablet_manager.cc:1398] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:54.173127 7716 tablet_bootstrap.cc:492] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:54.173931 7716 tablet_bootstrap.cc:654] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.174085 7712 tablet_bootstrap.cc:492] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:54.174166 7712 ts_tablet_manager.cc:1367] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 07:59:54.174584 7712 raft_consensus.cc:348] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.174698 7712 raft_consensus.cc:374] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.174734 7712 raft_consensus.cc:729] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.175086 7716 tablet_bootstrap.cc:492] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:54.175124 7712 consensus_queue.cc:260] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.175177 7716 ts_tablet_manager.cc:1367] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:54.175419 7712 ts_tablet_manager.cc:1398] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:54.175765 7716 raft_consensus.cc:348] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.175868 7716 raft_consensus.cc:374] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.175904 7716 raft_consensus.cc:729] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.176074 7716 consensus_queue.cc:260] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.176368 7716 ts_tablet_manager.cc:1398] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:54.177748 7711 tablet_bootstrap.cc:492] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:54.177853 7711 ts_tablet_manager.cc:1367] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240501 07:59:54.178314 7711 raft_consensus.cc:348] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.178429 7711 raft_consensus.cc:374] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.178464 7711 raft_consensus.cc:729] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.178609 7711 consensus_queue.cc:260] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.178898 7711 ts_tablet_manager.cc:1398] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:54.179116 7711 tablet_bootstrap.cc:492] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:54.179975 7711 tablet_bootstrap.cc:654] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.181488 7711 tablet_bootstrap.cc:492] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:54.181599 7711 ts_tablet_manager.cc:1367] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:54.182015 7711 raft_consensus.cc:348] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.182119 7711 raft_consensus.cc:374] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.182140 7711 raft_consensus.cc:729] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.182246 7711 consensus_queue.cc:260] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.182297 7581 raft_consensus.cc:482] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:54.182387 7581 raft_consensus.cc:504] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.182508 7711 ts_tablet_manager.cc:1398] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:54.182758 7581 leader_election.cc:288] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:54.183097 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbc4b418e4c5493fa8a7734740225200" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:54.183204 6201 raft_consensus.cc:2454] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 0.
I20240501 07:59:54.183187 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbc4b418e4c5493fa8a7734740225200" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:54.183320 6462 raft_consensus.cc:2454] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 0.
I20240501 07:59:54.183529 6530 leader_election.cc:302] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 953b945e3e7e4c71a61aa99c1208ab1d, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:54.184088 7581 raft_consensus.cc:2781] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:54.184150 7581 raft_consensus.cc:482] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:54.184187 7581 raft_consensus.cc:3037] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.185073 7581 raft_consensus.cc:504] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.185415 7581 leader_election.cc:288] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:54.185762 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbc4b418e4c5493fa8a7734740225200" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:54.185789 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbc4b418e4c5493fa8a7734740225200" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:54.185883 6201 raft_consensus.cc:3037] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.185895 6462 raft_consensus.cc:3037] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.186559 6201 raft_consensus.cc:2454] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 1.
I20240501 07:59:54.186707 6462 raft_consensus.cc:2454] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 1.
I20240501 07:59:54.187050 6530 leader_election.cc:302] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:54.187196 7581 raft_consensus.cc:2781] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:54.187287 7581 raft_consensus.cc:686] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Becoming Leader. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Running, Role: LEADER
I20240501 07:59:54.187418 7581 consensus_queue.cc:237] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.189023 6056 catalog_manager.cc:5484] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:54.269515 7692 raft_consensus.cc:482] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:54.269670 7692 raft_consensus.cc:504] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.270098 7692 leader_election.cc:288] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:54.270598 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98d43e2703b14f9ebf926b5f2e2b2c56" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:54.270658 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98d43e2703b14f9ebf926b5f2e2b2c56" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:54.270767 6201 raft_consensus.cc:2454] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:54.270792 6331 raft_consensus.cc:2454] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:54.271067 6399 leader_election.cc:302] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:54.271255 7692 raft_consensus.cc:2781] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:54.271319 7692 raft_consensus.cc:482] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:54.271358 7692 raft_consensus.cc:3037] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.272359 7692 raft_consensus.cc:504] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.272720 7692 leader_election.cc:288] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Requested vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:54.273147 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98d43e2703b14f9ebf926b5f2e2b2c56" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:54.273207 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98d43e2703b14f9ebf926b5f2e2b2c56" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:54.273285 6201 raft_consensus.cc:3037] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.273324 6331 raft_consensus.cc:3037] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.274255 6201 raft_consensus.cc:2454] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:54.274255 6331 raft_consensus.cc:2454] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:54.274575 6399 leader_election.cc:302] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 07:59:54.274753 7692 raft_consensus.cc:2781] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:54.274843 7692 raft_consensus.cc:686] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Becoming Leader. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Running, Role: LEADER
I20240501 07:59:54.274991 7692 consensus_queue.cc:237] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.276823 6056 catalog_manager.cc:5484] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } }
I20240501 07:59:54.303650 7581 raft_consensus.cc:482] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:54.303768 7581 raft_consensus.cc:504] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.304116 7581 leader_election.cc:288] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:54.304512 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96386dd2c62e45859e1f3b30510e8969" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:54.304545 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96386dd2c62e45859e1f3b30510e8969" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:54.304651 6201 raft_consensus.cc:2454] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 0.
I20240501 07:59:54.304672 6331 raft_consensus.cc:2454] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 0.
I20240501 07:59:54.304916 6530 leader_election.cc:302] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:54.305159 7581 raft_consensus.cc:2781] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:54.305220 7581 raft_consensus.cc:482] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:54.305248 7581 raft_consensus.cc:3037] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.306046 7581 raft_consensus.cc:504] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.306361 7581 leader_election.cc:288] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 election: Requested vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:54.306699 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96386dd2c62e45859e1f3b30510e8969" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:54.306733 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96386dd2c62e45859e1f3b30510e8969" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:54.306808 6201 raft_consensus.cc:3037] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.306850 6331 raft_consensus.cc:3037] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.307580 6201 raft_consensus.cc:2454] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 1.
I20240501 07:59:54.307586 6331 raft_consensus.cc:2454] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 1.
I20240501 07:59:54.307850 6530 leader_election.cc:302] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 953b945e3e7e4c71a61aa99c1208ab1d, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:54.308105 7581 raft_consensus.cc:2781] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:54.308187 7581 raft_consensus.cc:686] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Becoming Leader. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Running, Role: LEADER
I20240501 07:59:54.308326 7581 consensus_queue.cc:237] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.310081 6056 catalog_manager.cc:5484] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:54.421717 7727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:54.421970 7727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:54.425001 7727 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240501 07:59:54.517208 6462 raft_consensus.cc:1261] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.517632 7581 consensus_queue.cc:1035] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.517843 6200 raft_consensus.cc:1261] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.518590 7581 consensus_queue.cc:1035] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.519304 6331 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.519726 7692 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.523531 6331 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.523842 6201 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.523983 7581 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.524438 7653 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.552273 6201 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.552964 7691 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 122.74 ms
time per row: 0.06137 ms
I20240501 07:59:54.590620 6054 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43232:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 07:59:54.596150 6311 tablet_service.cc:1467] Processing CreateTablet for tablet de48f8c2870f4517b5349c30a51fce56 (DEFAULT_TABLE table=rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.596233 6572 tablet_service.cc:1467] Processing CreateTablet for tablet de48f8c2870f4517b5349c30a51fce56 (DEFAULT_TABLE table=rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.596272 6442 tablet_service.cc:1467] Processing CreateTablet for tablet de48f8c2870f4517b5349c30a51fce56 (DEFAULT_TABLE table=rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.596477 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de48f8c2870f4517b5349c30a51fce56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.596477 6572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de48f8c2870f4517b5349c30a51fce56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.596554 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de48f8c2870f4517b5349c30a51fce56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.596554 6441 tablet_service.cc:1467] Processing CreateTablet for tablet 88fa1c4aaf754e32bef123157b3bc62c (DEFAULT_TABLE table=rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.596767 6441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88fa1c4aaf754e32bef123157b3bc62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.598543 7716 tablet_bootstrap.cc:492] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:54.598598 7712 tablet_bootstrap.cc:492] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:54.599308 7712 tablet_bootstrap.cc:654] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.599309 7716 tablet_bootstrap.cc:654] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.599826 7713 tablet_bootstrap.cc:492] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:54.600626 7713 tablet_bootstrap.cc:654] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.600584 6181 tablet_service.cc:1467] Processing CreateTablet for tablet d2a4fd236f394ba0ad1812c50b7e66a3 (DEFAULT_TABLE table=rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.600725 6442 tablet_service.cc:1467] Processing CreateTablet for tablet d2a4fd236f394ba0ad1812c50b7e66a3 (DEFAULT_TABLE table=rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.600853 6181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2a4fd236f394ba0ad1812c50b7e66a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.600926 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2a4fd236f394ba0ad1812c50b7e66a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.600931 6310 tablet_service.cc:1467] Processing CreateTablet for tablet d2a4fd236f394ba0ad1812c50b7e66a3 (DEFAULT_TABLE table=rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.601155 6310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2a4fd236f394ba0ad1812c50b7e66a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.601694 7716 tablet_bootstrap.cc:492] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:54.601773 7716 ts_tablet_manager.cc:1367] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:54.602226 7716 raft_consensus.cc:348] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.602329 7716 raft_consensus.cc:374] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.602358 7716 raft_consensus.cc:729] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.602480 7716 consensus_queue.cc:260] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.602753 7716 ts_tablet_manager.cc:1398] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:54.602950 7711 tablet_bootstrap.cc:492] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d: Bootstrap starting.
I20240501 07:59:54.603087 6572 tablet_service.cc:1467] Processing CreateTablet for tablet 88fa1c4aaf754e32bef123157b3bc62c (DEFAULT_TABLE table=rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.603267 6572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88fa1c4aaf754e32bef123157b3bc62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.603744 7711 tablet_bootstrap.cc:654] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.604694 7711 tablet_bootstrap.cc:492] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d: No bootstrap required, opened a new log
I20240501 07:59:54.604772 7711 ts_tablet_manager.cc:1367] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:54.605080 7711 raft_consensus.cc:348] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.605194 7716 tablet_bootstrap.cc:492] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4: Bootstrap starting.
I20240501 07:59:54.605194 7711 raft_consensus.cc:374] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.605228 7711 raft_consensus.cc:729] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.605284 7712 tablet_bootstrap.cc:492] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:54.605362 7712 ts_tablet_manager.cc:1367] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240501 07:59:54.605338 7711 consensus_queue.cc:260] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.605592 7711 ts_tablet_manager.cc:1398] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:54.605777 7713 tablet_bootstrap.cc:492] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:54.605854 7713 ts_tablet_manager.cc:1367] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 07:59:54.605809 7712 raft_consensus.cc:348] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.605896 7712 raft_consensus.cc:374] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.605926 7712 raft_consensus.cc:729] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.606005 7716 tablet_bootstrap.cc:654] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.606040 7712 consensus_queue.cc:260] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.606289 7712 ts_tablet_manager.cc:1398] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:54.606271 7713 raft_consensus.cc:348] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.606361 7713 raft_consensus.cc:374] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.606388 7713 raft_consensus.cc:729] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.606451 7712 tablet_bootstrap.cc:492] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:54.606495 7713 consensus_queue.cc:260] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.606748 7713 ts_tablet_manager.cc:1398] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:54.606917 7713 tablet_bootstrap.cc:492] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:54.607268 7712 tablet_bootstrap.cc:654] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.607733 7716 tablet_bootstrap.cc:492] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4: No bootstrap required, opened a new log
I20240501 07:59:54.607760 7713 tablet_bootstrap.cc:654] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.607869 7716 ts_tablet_manager.cc:1367] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:54.608767 7716 raft_consensus.cc:348] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.608891 7716 raft_consensus.cc:374] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.608927 7716 raft_consensus.cc:729] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.609050 7716 consensus_queue.cc:260] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.609287 7716 ts_tablet_manager.cc:1398] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:54.610123 6311 tablet_service.cc:1467] Processing CreateTablet for tablet 88fa1c4aaf754e32bef123157b3bc62c (DEFAULT_TABLE table=rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 07:59:54.610355 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88fa1c4aaf754e32bef123157b3bc62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:54.612449 7712 tablet_bootstrap.cc:492] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:54.612538 7712 ts_tablet_manager.cc:1367] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 07:59:54.612947 7712 raft_consensus.cc:348] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.613055 7712 raft_consensus.cc:374] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.613076 7712 raft_consensus.cc:729] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.613166 7712 consensus_queue.cc:260] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.613395 7712 ts_tablet_manager.cc:1398] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:54.613852 7712 tablet_bootstrap.cc:492] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4: Bootstrap starting.
I20240501 07:59:54.614442 7713 tablet_bootstrap.cc:492] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:54.614535 7713 ts_tablet_manager.cc:1367] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240501 07:59:54.614740 7712 tablet_bootstrap.cc:654] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.614992 7713 raft_consensus.cc:348] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.615093 7713 raft_consensus.cc:374] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.615123 7713 raft_consensus.cc:729] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.615268 7713 consensus_queue.cc:260] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.615630 7713 ts_tablet_manager.cc:1398] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 07:59:54.615810 7713 tablet_bootstrap.cc:492] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1: Bootstrap starting.
I20240501 07:59:54.616667 7713 tablet_bootstrap.cc:654] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1: Neither blocks nor log segments found. Creating new log.
I20240501 07:59:54.616675 7712 tablet_bootstrap.cc:492] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4: No bootstrap required, opened a new log
I20240501 07:59:54.616750 7712 ts_tablet_manager.cc:1367] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:54.617172 7712 raft_consensus.cc:348] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.617292 7712 raft_consensus.cc:374] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.617322 7712 raft_consensus.cc:729] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.617444 7712 consensus_queue.cc:260] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.617720 7713 tablet_bootstrap.cc:492] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1: No bootstrap required, opened a new log
I20240501 07:59:54.617761 7712 ts_tablet_manager.cc:1398] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:54.617869 7713 ts_tablet_manager.cc:1367] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:54.619283 7713 raft_consensus.cc:348] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.619403 7713 raft_consensus.cc:374] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 07:59:54.619436 7713 raft_consensus.cc:729] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Initialized, Role: FOLLOWER
I20240501 07:59:54.619561 7713 consensus_queue.cc:260] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.619850 7713 ts_tablet_manager.cc:1398] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240501 07:59:54.647759 7759 raft_consensus.cc:482] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:54.647913 7759 raft_consensus.cc:504] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.648365 7759 leader_election.cc:288] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 07:59:54.648687 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de48f8c2870f4517b5349c30a51fce56" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:54.648761 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de48f8c2870f4517b5349c30a51fce56" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:54.648824 6592 raft_consensus.cc:2454] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:54.648900 6462 raft_consensus.cc:2454] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:54.649137 6268 leader_election.cc:302] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:54.649302 7759 raft_consensus.cc:2781] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:54.649359 7759 raft_consensus.cc:482] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:54.649403 7759 raft_consensus.cc:3037] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.650184 7759 raft_consensus.cc:504] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.650481 7759 leader_election.cc:288] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Requested vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 07:59:54.650787 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de48f8c2870f4517b5349c30a51fce56" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:54.650897 6592 raft_consensus.cc:3037] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.650874 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de48f8c2870f4517b5349c30a51fce56" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:54.650972 6462 raft_consensus.cc:3037] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.651667 6462 raft_consensus.cc:2454] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:54.651855 6592 raft_consensus.cc:2454] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:54.651912 6268 leader_election.cc:302] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:54.652130 7759 raft_consensus.cc:2781] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:54.652228 7759 raft_consensus.cc:686] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Becoming Leader. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Running, Role: LEADER
I20240501 07:59:54.652366 7759 consensus_queue.cc:237] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } }
I20240501 07:59:54.654271 6055 catalog_manager.cc:5484] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a155eaa444d440d90fde420951458e4 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:54.656165 7692 raft_consensus.cc:482] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:54.656265 7692 raft_consensus.cc:504] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.656589 7692 leader_election.cc:288] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:54.656957 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2a4fd236f394ba0ad1812c50b7e66a3" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4" is_pre_election: true
I20240501 07:59:54.656962 6198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2a4fd236f394ba0ad1812c50b7e66a3" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" is_pre_election: true
I20240501 07:59:54.657066 6331 raft_consensus.cc:2454] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:54.657080 6198 raft_consensus.cc:2454] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 0.
I20240501 07:59:54.657358 6399 leader_election.cc:302] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:54.657505 7692 raft_consensus.cc:2781] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:54.657560 7692 raft_consensus.cc:482] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:54.657604 7692 raft_consensus.cc:3037] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.658522 7692 raft_consensus.cc:504] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.658865 7692 leader_election.cc:288] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Requested vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:54.659144 6198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2a4fd236f394ba0ad1812c50b7e66a3" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 07:59:54.659191 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2a4fd236f394ba0ad1812c50b7e66a3" candidate_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 07:59:54.659267 6198 raft_consensus.cc:3037] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.659287 6331 raft_consensus.cc:3037] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.660183 6331 raft_consensus.cc:2454] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:54.660189 6198 raft_consensus.cc:2454] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db2970f4bbd4a3c92d7d913f99716b1 in term 1.
I20240501 07:59:54.660470 6399 leader_election.cc:302] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:54.660719 7692 raft_consensus.cc:2781] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:54.660801 7692 raft_consensus.cc:686] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Becoming Leader. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Running, Role: LEADER
I20240501 07:59:54.660925 7692 consensus_queue.cc:237] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } }
I20240501 07:59:54.662683 6055 catalog_manager.cc:5484] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:54.745733 7759 raft_consensus.cc:482] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 07:59:54.745864 7759 raft_consensus.cc:504] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.746302 7759 leader_election.cc:288] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:54.746680 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88fa1c4aaf754e32bef123157b3bc62c" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" is_pre_election: true
I20240501 07:59:54.746831 6462 raft_consensus.cc:2454] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:54.746865 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88fa1c4aaf754e32bef123157b3bc62c" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4" is_pre_election: true
I20240501 07:59:54.746977 6593 raft_consensus.cc:2454] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 0.
I20240501 07:59:54.747114 6268 leader_election.cc:302] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 07:59:54.747304 7759 raft_consensus.cc:2781] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 07:59:54.747373 7759 raft_consensus.cc:482] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 07:59:54.747418 7759 raft_consensus.cc:3037] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.748379 7759 raft_consensus.cc:504] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.748672 7759 leader_election.cc:288] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:54.748996 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88fa1c4aaf754e32bef123157b3bc62c" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 07:59:54.749104 6462 raft_consensus.cc:3037] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.749073 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88fa1c4aaf754e32bef123157b3bc62c" candidate_uuid: "4a155eaa444d440d90fde420951458e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 07:59:54.749186 6593 raft_consensus.cc:3037] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 0 FOLLOWER]: Advancing to term 1
I20240501 07:59:54.749907 6593 raft_consensus.cc:2454] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:54.749912 6462 raft_consensus.cc:2454] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a155eaa444d440d90fde420951458e4 in term 1.
I20240501 07:59:54.750167 6268 leader_election.cc:302] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 07:59:54.750327 7759 raft_consensus.cc:2781] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 07:59:54.750414 7759 raft_consensus.cc:686] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Becoming Leader. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Running, Role: LEADER
I20240501 07:59:54.750547 7759 consensus_queue.cc:237] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } }
I20240501 07:59:54.752395 6055 catalog_manager.cc:5484] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a155eaa444d440d90fde420951458e4 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:54.867842 7779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:54.868145 7779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:54.871143 7779 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240501 07:59:54.949239 6331 raft_consensus.cc:1261] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.951514 6462 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.951583 6593 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.951583 6594 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.951730 7692 consensus_queue.cc:1035] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.952183 6463 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.952396 7759 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.952766 6198 raft_consensus.cc:1261] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 07:59:54.953189 7692 consensus_queue.cc:1035] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.952410 7671 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20240501 07:59:54.959815 7759 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.960392 7759 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 07:59:54.972151 7693 mvcc.cc:205] Tried to move back new op lower bound from 7022798417705857024 to 7022798416532353024. Current Snapshot: MvccSnapshot[applied={T|T < 7022798417705857024}]
I20240501 07:59:55.002446 7761 mvcc.cc:205] Tried to move back new op lower bound from 7022798417714094080 to 7022798416497180672. Current Snapshot: MvccSnapshot[applied={T|T < 7022798417714094080 or (T in {7022798417714094080})}]
INSERT report
rows total: 2000
time total: 107.684 ms
time per row: 0.0538418 ms
I20240501 07:59:55.021821 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:39247
--local_ip_for_outbound_sockets=127.0.99.133
--tserver_master_addrs=127.0.99.190:41421
--webserver_port=39035
--webserver_interface=127.0.99.133
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240501 07:59:55.132714 7818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:55.133023 7818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:55.133124 7818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:55.136432 7818 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:55.136499 7818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:55.136586 7818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240501 07:59:55.140724 7818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:39247
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=39035
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.7818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:55.141914 7818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:55.143394 7818 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240501 07:59:55.154917 7818 server_base.cc:1034] running on GCE node
W20240501 07:59:55.155360 7824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:55.155853 7826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:55.156391 7823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:55.156942 7818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:55.157707 7818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:55.158883 7818 hybrid_clock.cc:648] HybridClock initialized: now 1714550395158856 us; error 53 us; skew 500 ppm
I20240501 07:59:55.161531 7818 webserver.cc:460] Webserver started at http://127.0.99.133:39035/ using document root <none> and password file <none>
I20240501 07:59:55.162276 7818 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:55.162370 7818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:55.166714 7818 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 07:59:55.168805 7832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:55.169896 7818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:55.170023 7818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "7a80bdedf813425faf7617b4b07c9f36"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:55.170459 7818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:55.187814 7818 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:55.188552 7818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:55.188704 7818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:55.189254 7818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:55.190146 7818 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:55.190191 7818 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:55.190241 7818 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:55.190269 7818 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:55.198505 7818 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:39247
I20240501 07:59:55.198522 7945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:39247 every 8 connection(s)
I20240501 07:59:55.199573 7818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 07:59:55.207455 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 7818
I20240501 07:59:55.207891 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.134:46293
--local_ip_for_outbound_sockets=127.0.99.134
--tserver_master_addrs=127.0.99.190:41421
--webserver_port=37409
--webserver_interface=127.0.99.134
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 07:59:55.208155 7946 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:55.208395 7946 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:55.209039 7946 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:55.210314 6053 ts_manager.cc:194] Re-registered known tserver with Master: 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247)
I20240501 07:59:55.211028 6053 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:44561
W20240501 07:59:55.319074 7949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:55.319325 7949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:55.319373 7949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:55.322650 7949 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:55.322728 7949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:55.322803 7949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.134
I20240501 07:59:55.326648 7949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.134:46293
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.0.99.134
--webserver_port=37409
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.7949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.134
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:55.327551 7949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:55.328747 7949 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:55.337232 7954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:55.337694 7955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:55.338276 7957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:55.338812 7949 server_base.cc:1034] running on GCE node
I20240501 07:59:55.339229 7949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:55.339799 7949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:55.340950 7949 hybrid_clock.cc:648] HybridClock initialized: now 1714550395340932 us; error 33 us; skew 500 ppm
I20240501 07:59:55.346755 7949 webserver.cc:460] Webserver started at http://127.0.99.134:37409/ using document root <none> and password file <none>
I20240501 07:59:55.347321 7949 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:55.347385 7949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:55.350749 7949 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:55.352783 7963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:55.353828 7949 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:55.353938 7949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "29afadb287c049d9bb777d674b3697ce"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:55.354420 7949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:55.388250 7949 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:55.388864 7949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:55.389003 7949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:55.389518 7949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:55.390398 7949 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:55.390441 7949 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:55.390482 7949 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:55.390501 7949 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:55.398885 7949 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.134:46293
I20240501 07:59:55.398922 8076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.134:46293 every 8 connection(s)
I20240501 07:59:55.399981 7949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240501 07:59:55.404497 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 7949
I20240501 07:59:55.404896 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.135:34937
--local_ip_for_outbound_sockets=127.0.99.135
--tserver_master_addrs=127.0.99.190:41421
--webserver_port=43095
--webserver_interface=127.0.99.135
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 07:59:55.409125 8077 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:55.409404 8077 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:55.410143 8077 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:55.411298 6053 ts_manager.cc:194] Re-registered known tserver with Master: 29afadb287c049d9bb777d674b3697ce (127.0.99.134:46293)
I20240501 07:59:55.412142 6053 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.134:54083
W20240501 07:59:55.520115 8080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:55.520355 8080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:55.520404 8080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:55.523600 8080 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:55.523671 8080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:55.523743 8080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.135
I20240501 07:59:55.527616 8080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.135:34937
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.0.99.135
--webserver_port=43095
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.8080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.135
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:55.528641 8080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:55.529785 8080 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:55.538889 8088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:55.539364 8085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:55.539954 8086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:55.540506 8080 server_base.cc:1034] running on GCE node
I20240501 07:59:55.540958 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:55.541550 8080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:55.542706 8080 hybrid_clock.cc:648] HybridClock initialized: now 1714550395542681 us; error 39 us; skew 500 ppm
I20240501 07:59:55.544688 8080 webserver.cc:460] Webserver started at http://127.0.99.135:43095/ using document root <none> and password file <none>
I20240501 07:59:55.545240 8080 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:55.545313 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:55.548871 8080 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20240501 07:59:55.550920 8094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:55.552103 8080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 07:59:55.552242 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "ef83010b77b441ffac2e41468a0a1bd1"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:55.552659 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:55.570235 8080 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:55.570902 8080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:55.571053 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:55.571581 8080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:55.572501 8080 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:55.572548 8080 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:55.572602 8080 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:55.572625 8080 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:55.581256 8080 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.135:34937
I20240501 07:59:55.581360 8207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.135:34937 every 8 connection(s)
I20240501 07:59:55.582293 8080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 07:59:55.591549 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 8080
I20240501 07:59:55.592073 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.136:40449
--local_ip_for_outbound_sockets=127.0.99.136
--tserver_master_addrs=127.0.99.190:41421
--webserver_port=36805
--webserver_interface=127.0.99.136
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 07:59:55.592821 8208 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:55.593060 8208 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:55.594053 8208 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:55.595419 6053 ts_manager.cc:194] Re-registered known tserver with Master: ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937)
I20240501 07:59:55.596338 6053 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.135:53841
W20240501 07:59:55.704414 8212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:55.704643 8212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:55.704694 8212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:55.707881 8212 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:55.707986 8212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:55.708091 8212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.136
I20240501 07:59:55.712015 8212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.136:40449
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.0.99.136
--webserver_port=36805
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.8212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.136
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:55.712963 8212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:55.714215 8212 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 07:59:55.723248 8217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:55.723718 8220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:55.724366 8218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:55.724946 8212 server_base.cc:1034] running on GCE node
I20240501 07:59:55.725442 8212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:55.726023 8212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:55.727211 8212 hybrid_clock.cc:648] HybridClock initialized: now 1714550395727185 us; error 44 us; skew 500 ppm
I20240501 07:59:55.734135 8212 webserver.cc:460] Webserver started at http://127.0.99.136:36805/ using document root <none> and password file <none>
I20240501 07:59:55.734853 8212 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:55.735013 8212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:55.738926 8212 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240501 07:59:55.741176 8226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:55.742482 8212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:55.742591 8212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "06c63579c7234194b5f6ef457dd5e3f8"
format_stamp: "Formatted at 2024-05-01 07:59:45 on dist-test-slave-f81v"
I20240501 07:59:55.743108 8212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:55.767824 8212 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:55.768515 8212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:55.768671 8212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:55.769227 8212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:55.770124 8212 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:55.770169 8212 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:55.770215 8212 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:55.770242 8212 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:55.778633 8212 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.136:40449
I20240501 07:59:55.778672 8339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.136:40449 every 8 connection(s)
I20240501 07:59:55.779716 8212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 07:59:55.780874 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 8212
I20240501 07:59:55.791049 8340 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:55.791324 8340 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:55.792093 8340 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:55.793349 6053 ts_manager.cc:194] Re-registered known tserver with Master: 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136:40449)
I20240501 07:59:55.794090 6053 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.136:57303
W20240501 07:59:55.901507 8344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:55.901870 8344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:55.906198 8344 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 07:59:56.031809 8011 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.032693 6311 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 07:59:56.042137 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.043193 6441 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 07:59:56.056762 6572 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 07:59:56.057304 7880 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.079008 6181 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 07:59:56.080248 8142 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.090691 6441 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 07:59:56.091809 8011 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.093536 7880 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.094391 6311 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 07:59:56.116731 6572 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 07:59:56.121959 6181 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 07:59:56.124611 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.145709 8142 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.213207 7946 heartbeater.cc:503] Master 127.0.99.190:41421 was elected leader, sending a full tablet report...
I20240501 07:59:56.268718 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.271139 6311 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 07:59:56.273772 6441 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 07:59:56.275482 8011 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.282092 7880 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.289373 8142 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.296010 6181 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 07:59:56.320919 6572 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
W20240501 07:59:56.344621 8457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:56.344880 8457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:56.348098 8457 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 07:59:56.414714 8077 heartbeater.cc:503] Master 127.0.99.190:41421 was elected leader, sending a full tablet report...
I20240501 07:59:56.459656 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.461807 8011 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.465618 7880 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.465870 8142 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.476677 6311 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 07:59:56.481436 6181 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 07:59:56.483695 6441 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 07:59:56.502756 6572 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 07:59:56.542486 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.547720 7880 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.549360 6311 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 07:59:56.558374 6572 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 07:59:56.560928 8011 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.562893 8142 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.570034 6441 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 07:59:56.570916 6181 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 07:59:56.598464 8208 heartbeater.cc:503] Master 127.0.99.190:41421 was elected leader, sending a full tablet report...
I20240501 07:59:56.636938 6329 consensus_queue.cc:237] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:56.639097 6463 raft_consensus.cc:1261] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.639143 6201 raft_consensus.cc:1261] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.639634 8582 consensus_queue.cc:1035] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.639896 8400 consensus_queue.cc:1035] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.642268 8582 raft_consensus.cc:2932] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
W20240501 07:59:56.642865 6265 consensus_peers.cc:485] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 -> Peer 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136:40449): Couldn't send request to peer 06c63579c7234194b5f6ef457dd5e3f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3098236442b242b38c5937de465c8e1f. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.642993 6463 raft_consensus.cc:2932] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:56.645773 6056 catalog_manager.cc:5484] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:56.653110 6201 raft_consensus.cc:2932] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:56.656952 6593 consensus_queue.cc:237] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:56.658619 6201 raft_consensus.cc:1261] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.658617 6463 raft_consensus.cc:1261] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.659229 8596 consensus_queue.cc:1035] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.659447 8489 consensus_queue.cc:1035] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.661347 7653 raft_consensus.cc:2932] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:56.661670 6201 raft_consensus.cc:2932] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:56.662961 6463 raft_consensus.cc:2932] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:56.665202 6057 catalog_manager.cc:5484] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:56.665395 6529 consensus_peers.cc:485] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 -> Peer 29afadb287c049d9bb777d674b3697ce (127.0.99.134:46293): Couldn't send request to peer 29afadb287c049d9bb777d674b3697ce. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a9dcf4944c34707a8cd0acebc8f9029. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.675906 6201 consensus_queue.cc:237] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:56.677784 6463 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.677915 6593 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.678221 8607 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.678597 8607 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.680148 8607 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:56.682579 6463 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:56.683176 6593 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:56.683569 6056 catalog_manager.cc:5484] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:56.691825 6135 consensus_peers.cc:485] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d -> Peer ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937): Couldn't send request to peer ef83010b77b441ffac2e41468a0a1bd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16752f91ea894a7ba4017170c1dc6ec1. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.694540 6463 consensus_queue.cc:237] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:56.696139 6329 raft_consensus.cc:1261] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.696489 6201 raft_consensus.cc:1261] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.696552 8617 consensus_queue.cc:1035] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.697693 8617 consensus_queue.cc:1035] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.698282 7691 raft_consensus.cc:2932] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.700342 6329 raft_consensus.cc:2932] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.701479 6055 catalog_manager.cc:5484] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:56.705020 6397 consensus_peers.cc:485] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247): Couldn't send request to peer 7a80bdedf813425faf7617b4b07c9f36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d2a4fd236f394ba0ad1812c50b7e66a3. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.706880 6201 raft_consensus.cc:2932] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.727315 6463 consensus_queue.cc:237] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:56.730127 6329 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.730554 8211 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.731514 6201 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.731999 8211 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 07:59:56.733290 8575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20240501 07:59:56.733480 8211 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
W20240501 07:59:56.733671 8575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:56.733995 6396 consensus_peers.cc:485] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136:40449): Couldn't send request to peer 06c63579c7234194b5f6ef457dd5e3f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e60dad8ddd2402faae35be18662c8c3. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.734490 6329 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:56.736574 6057 catalog_manager.cc:5484] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:56.737797 8575 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 07:59:56.739187 6201 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:56.748562 6329 consensus_queue.cc:237] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:56.750243 6462 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.750243 6593 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.752179 7817 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.752440 8582 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 07:59:56.753720 6266 consensus_peers.cc:485] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 -> Peer ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937): Couldn't send request to peer ef83010b77b441ffac2e41468a0a1bd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8bf587ac1d8141e99fceab28d89e4ea9. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.755007 7817 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:56.756471 6462 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:56.756758 6593 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:56.758754 6057 catalog_manager.cc:5484] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:56.768437 6329 consensus_queue.cc:237] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:56.769897 6462 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.769892 6593 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.770347 7817 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.770715 7817 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 07:59:56.774137 6267 consensus_peers.cc:485] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 -> Peer 29afadb287c049d9bb777d674b3697ce (127.0.99.134:46293): Couldn't send request to peer 29afadb287c049d9bb777d674b3697ce. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41b9f1a284c34a9099cf1014db77f35c. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.775605 8400 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:56.776268 6462 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:56.778292 6593 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:56.780658 6055 catalog_manager.cc:5484] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:56.794405 6462 consensus_queue.cc:237] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:56.796015 6329 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.796120 6593 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:56.796188 6397 consensus_peers.cc:485] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247): Couldn't send request to peer 7a80bdedf813425faf7617b4b07c9f36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f24f629734649a3b31c7e293122d7ce. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.796684 8340 heartbeater.cc:503] Master 127.0.99.190:41421 was elected leader, sending a full tablet report...
I20240501 07:59:56.796885 8211 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.797601 7691 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.799638 8617 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.799849 6593 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.799875 6329 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.802490 6055 catalog_manager.cc:5484] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.816397 6329 consensus_queue.cc:237] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:56.818113 6593 raft_consensus.cc:1261] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:56.818390 6266 consensus_peers.cc:485] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 -> Peer ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937): Couldn't send request to peer ef83010b77b441ffac2e41468a0a1bd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5a31c7d3de834d1aab1ffa16e6df7e15. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.818509 7817 consensus_queue.cc:1035] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.819636 6201 raft_consensus.cc:1261] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.822407 8582 consensus_queue.cc:1035] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.823009 8582 raft_consensus.cc:2932] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:56.825775 6051 catalog_manager.cc:5484] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:56.826120 6593 raft_consensus.cc:2932] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:56.835610 6201 raft_consensus.cc:2932] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:56.839725 6329 consensus_queue.cc:237] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
W20240501 07:59:56.841490 6265 consensus_peers.cc:485] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 -> Peer 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136:40449): Couldn't send request to peer 06c63579c7234194b5f6ef457dd5e3f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 88fa1c4aaf754e32bef123157b3bc62c. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.843072 6593 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.843575 6463 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.843648 7817 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.845149 7817 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.846315 7817 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:56.848469 6593 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:56.849004 6052 catalog_manager.cc:5484] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:56.850832 6463 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:56.861611 8011 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.864315 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.865747 6441 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 07:59:56.871009 7880 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.878538 6201 consensus_queue.cc:237] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:56.880808 6593 raft_consensus.cc:1261] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.881276 6463 raft_consensus.cc:1261] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.881736 8606 consensus_queue.cc:1035] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.882122 8606 consensus_queue.cc:1035] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 07:59:56.885064 6135 consensus_peers.cc:485] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d -> Peer 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247): Couldn't send request to peer 7a80bdedf813425faf7617b4b07c9f36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5d0ae7c274af4ce1992e9507cbabc551. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.891356 8142 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:56.895741 6181 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 07:59:56.896847 8607 raft_consensus.cc:2932] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.898114 6311 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 07:59:56.900135 6056 catalog_manager.cc:5484] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:56.900394 6593 raft_consensus.cc:2932] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.912542 6463 raft_consensus.cc:2932] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.918856 6572 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 07:59:56.930920 6463 consensus_queue.cc:237] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:56.934064 6201 raft_consensus.cc:1261] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.934827 6329 raft_consensus.cc:1261] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.935878 8617 consensus_queue.cc:1035] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 07:59:56.937243 6398 consensus_peers.cc:485] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer 29afadb287c049d9bb777d674b3697ce (127.0.99.134:46293): Couldn't send request to peer 29afadb287c049d9bb777d674b3697ce. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 47d3021e3ac74c6ba482b8b54066c36e. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.937772 8211 consensus_queue.cc:1035] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.938520 7691 raft_consensus.cc:2932] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:56.940677 6201 raft_consensus.cc:2932] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:56.940923 6329 raft_consensus.cc:2932] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:56.941321 6056 catalog_manager.cc:5484] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:56.952318 6201 consensus_queue.cc:237] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
Runtime error: ksck discovered errors
W20240501 07:59:56.954485 6135 consensus_peers.cc:485] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d -> Peer 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247): Couldn't send request to peer 7a80bdedf813425faf7617b4b07c9f36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5dae6c0b9c1d4009a93ed157e1f763c8. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.954566 6593 raft_consensus.cc:1261] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.955113 6329 raft_consensus.cc:1261] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.955343 7333 consensus_queue.cc:1035] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.955528 8606 consensus_queue.cc:1035] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.957268 8606 raft_consensus.cc:2932] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.959863 6056 catalog_manager.cc:5484] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:56.961378 6593 raft_consensus.cc:2932] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.962697 6329 raft_consensus.cc:2932] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:56.969616 6329 consensus_queue.cc:237] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:56.971072 6593 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.971089 6463 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:56.971395 6266 consensus_peers.cc:485] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 -> Peer ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937): Couldn't send request to peer ef83010b77b441ffac2e41468a0a1bd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f4c62996a5a64d978850f2f7a31009f4. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.971452 8582 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.971662 7817 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.973374 8582 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:56.973788 6593 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:56.973935 6463 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:56.976642 6056 catalog_manager.cc:5484] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:56.984899 6201 consensus_queue.cc:237] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:56.986801 6593 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.986809 6329 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:56.987301 7333 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.987660 7333 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:56.989882 8607 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
W20240501 07:59:56.990078 6134 consensus_peers.cc:485] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d -> Peer 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136:40449): Couldn't send request to peer 06c63579c7234194b5f6ef457dd5e3f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6b552a44c67940819f9ef64c68585eb1. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:56.990301 6329 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:56.992908 6052 catalog_manager.cc:5484] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:57.001432 6201 consensus_queue.cc:237] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:57.003086 6329 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:57.003333 6463 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:57.003459 8607 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:57.003767 8606 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:57.005327 6593 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:57.005673 8607 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
W20240501 07:59:57.007479 6136 consensus_peers.cc:485] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d -> Peer 29afadb287c049d9bb777d674b3697ce (127.0.99.134:46293): Couldn't send request to peer 29afadb287c049d9bb777d674b3697ce. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d929e6e09be943b68061ad4b1bb8e9f1. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:57.009302 6053 catalog_manager.cc:5484] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:57.009799 6463 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:57.015962 6329 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:57.019150 8765 ts_tablet_manager.cc:902] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:57.019958 8765 tablet_copy_client.cc:308] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 07:59:57.020962 6463 consensus_queue.cc:237] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:57.022558 6201 raft_consensus.cc:1261] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:57.022563 6593 raft_consensus.cc:1261] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:57.022667 6397 consensus_peers.cc:485] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247): Couldn't send request to peer 7a80bdedf813425faf7617b4b07c9f36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 947b5d16224a42dcad709e7ddefc0b33. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:57.022959 7691 consensus_queue.cc:1035] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:57.023149 8211 consensus_queue.cc:1035] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:57.024987 8211 raft_consensus.cc:2932] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:57.027045 6201 raft_consensus.cc:2932] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:57.027457 6056 catalog_manager.cc:5484] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:57.029661 6593 raft_consensus.cc:2932] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:57.038010 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet 3098236442b242b38c5937de465c8e1f from peer 06c63579c7234194b5f6ef457dd5e3f8 ({username='slave'} at 127.0.99.136:47239)
I20240501 07:59:57.038157 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet 3098236442b242b38c5937de465c8e1f from peer 06c63579c7234194b5f6ef457dd5e3f8 at {username='slave'} at 127.0.99.136:47239: session id = 06c63579c7234194b5f6ef457dd5e3f8-3098236442b242b38c5937de465c8e1f
I20240501 07:59:57.039533 6352 tablet_copy_source_session.cc:205] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.041821 6329 consensus_queue.cc:237] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:57.042754 8765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3098236442b242b38c5937de465c8e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.043531 6593 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:57.043629 6266 consensus_peers.cc:485] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 -> Peer ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937): Couldn't send request to peer ef83010b77b441ffac2e41468a0a1bd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de48f8c2870f4517b5349c30a51fce56. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:57.043907 6463 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:57.044481 7817 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:57.044682 8582 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:57.046797 8582 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:57.047336 6593 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:57.049345 8765 tablet_copy_client.cc:783] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.049542 8765 tablet_copy_client.cc:647] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.050043 6053 catalog_manager.cc:5484] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:57.052726 8767 tablet_copy_client.cc:1002] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.059720 6463 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:57.062662 6201 consensus_queue.cc:237] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:57.064335 6463 raft_consensus.cc:1261] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:57.064687 6134 consensus_peers.cc:485] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d -> Peer 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136:40449): Couldn't send request to peer 06c63579c7234194b5f6ef457dd5e3f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e952704c6a3d43aba833484bba9fddbf. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:57.064723 8607 consensus_queue.cc:1035] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:57.064785 6329 raft_consensus.cc:1261] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:57.065235 8607 consensus_queue.cc:1035] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:57.066547 8606 raft_consensus.cc:2932] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:57.067046 6463 raft_consensus.cc:2932] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:57.068409 6329 raft_consensus.cc:2932] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:57.069608 6052 catalog_manager.cc:5484] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:57.118271 8831 ts_tablet_manager.cc:902] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1: Initiating tablet copy from peer 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:57.119077 8831 tablet_copy_client.cc:308] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:44189
I20240501 07:59:57.125129 6221 tablet_copy_service.cc:140] P 953b945e3e7e4c71a61aa99c1208ab1d: Received BeginTabletCopySession request for tablet 16752f91ea894a7ba4017170c1dc6ec1 from peer ef83010b77b441ffac2e41468a0a1bd1 ({username='slave'} at 127.0.99.135:33435)
I20240501 07:59:57.125293 6221 tablet_copy_service.cc:161] P 953b945e3e7e4c71a61aa99c1208ab1d: Beginning new tablet copy session on tablet 16752f91ea894a7ba4017170c1dc6ec1 from peer ef83010b77b441ffac2e41468a0a1bd1 at {username='slave'} at 127.0.99.135:33435: session id = ef83010b77b441ffac2e41468a0a1bd1-16752f91ea894a7ba4017170c1dc6ec1
I20240501 07:59:57.127374 6221 tablet_copy_source_session.cc:205] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.131239 8831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16752f91ea894a7ba4017170c1dc6ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.137975 8831 tablet_copy_client.cc:783] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.138226 8831 tablet_copy_client.cc:647] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.140321 8835 tablet_copy_client.cc:1002] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Injecting latency into file download: 1500
W20240501 07:59:57.144650 8743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:57.144963 8743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 07:59:57.149036 8743 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 07:59:57.166224 8881 ts_tablet_manager.cc:902] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:57.166939 8881 tablet_copy_client.cc:308] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 07:59:57.171696 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet 41b9f1a284c34a9099cf1014db77f35c from peer 29afadb287c049d9bb777d674b3697ce ({username='slave'} at 127.0.99.134:51035)
I20240501 07:59:57.171815 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet 41b9f1a284c34a9099cf1014db77f35c from peer 29afadb287c049d9bb777d674b3697ce at {username='slave'} at 127.0.99.134:51035: session id = 29afadb287c049d9bb777d674b3697ce-41b9f1a284c34a9099cf1014db77f35c
I20240501 07:59:57.172796 6352 tablet_copy_source_session.cc:205] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.176299 8881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b9f1a284c34a9099cf1014db77f35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.180773 8898 ts_tablet_manager.cc:902] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 07:59:57.181504 8898 tablet_copy_client.cc:308] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
I20240501 07:59:57.187351 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet 6f24f629734649a3b31c7e293122d7ce from peer 7a80bdedf813425faf7617b4b07c9f36 ({username='slave'} at 127.0.99.133:60211)
I20240501 07:59:57.187505 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet 6f24f629734649a3b31c7e293122d7ce from peer 7a80bdedf813425faf7617b4b07c9f36 at {username='slave'} at 127.0.99.133:60211: session id = 7a80bdedf813425faf7617b4b07c9f36-6f24f629734649a3b31c7e293122d7ce
I20240501 07:59:57.188869 6483 tablet_copy_source_session.cc:205] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.192453 8881 tablet_copy_client.cc:783] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.192780 8881 tablet_copy_client.cc:647] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.194149 8898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f24f629734649a3b31c7e293122d7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.194643 8886 tablet_copy_client.cc:1002] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.202633 8898 tablet_copy_client.cc:783] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.202908 8898 tablet_copy_client.cc:647] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.205050 8900 tablet_copy_client.cc:1002] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.238718 8934 ts_tablet_manager.cc:902] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 07:59:57.239771 8934 tablet_copy_client.cc:308] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
I20240501 07:59:57.240257 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet d2a4fd236f394ba0ad1812c50b7e66a3 from peer 7a80bdedf813425faf7617b4b07c9f36 ({username='slave'} at 127.0.99.133:60211)
I20240501 07:59:57.240373 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet d2a4fd236f394ba0ad1812c50b7e66a3 from peer 7a80bdedf813425faf7617b4b07c9f36 at {username='slave'} at 127.0.99.133:60211: session id = 7a80bdedf813425faf7617b4b07c9f36-d2a4fd236f394ba0ad1812c50b7e66a3
I20240501 07:59:57.241255 6483 tablet_copy_source_session.cc:205] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.241393 8939 ts_tablet_manager.cc:902] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:57.242130 8934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2a4fd236f394ba0ad1812c50b7e66a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.243021 8939 tablet_copy_client.cc:308] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 07:59:57.248008 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet 8bf587ac1d8141e99fceab28d89e4ea9 from peer ef83010b77b441ffac2e41468a0a1bd1 ({username='slave'} at 127.0.99.135:48397)
I20240501 07:59:57.248121 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet 8bf587ac1d8141e99fceab28d89e4ea9 from peer ef83010b77b441ffac2e41468a0a1bd1 at {username='slave'} at 127.0.99.135:48397: session id = ef83010b77b441ffac2e41468a0a1bd1-8bf587ac1d8141e99fceab28d89e4ea9
I20240501 07:59:57.249051 6352 tablet_copy_source_session.cc:205] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.249363 8934 tablet_copy_client.cc:783] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.250159 8939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bf587ac1d8141e99fceab28d89e4ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.250326 8956 ts_tablet_manager.cc:902] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:57.250941 8956 tablet_copy_client.cc:308] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 07:59:57.251355 8934 tablet_copy_client.cc:647] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.252264 8939 tablet_copy_client.cc:783] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.252391 8939 tablet_copy_client.cc:647] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.253185 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet 5a31c7d3de834d1aab1ffa16e6df7e15 from peer ef83010b77b441ffac2e41468a0a1bd1 ({username='slave'} at 127.0.99.135:48397)
I20240501 07:59:57.253283 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet 5a31c7d3de834d1aab1ffa16e6df7e15 from peer ef83010b77b441ffac2e41468a0a1bd1 at {username='slave'} at 127.0.99.135:48397: session id = ef83010b77b441ffac2e41468a0a1bd1-5a31c7d3de834d1aab1ffa16e6df7e15
I20240501 07:59:57.253844 8940 tablet_copy_client.cc:1002] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.254235 6352 tablet_copy_source_session.cc:205] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.255744 8956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a31c7d3de834d1aab1ffa16e6df7e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.256299 7880 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:57.256932 8938 tablet_copy_client.cc:1002] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.259485 8956 tablet_copy_client.cc:783] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.259725 8956 tablet_copy_client.cc:647] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.259816 8011 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:57.260785 8961 tablet_copy_client.cc:1002] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.261060 8962 ts_tablet_manager.cc:902] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 07:59:57.261593 8962 tablet_copy_client.cc:308] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
I20240501 07:59:57.263849 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:57.266258 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet 4e60dad8ddd2402faae35be18662c8c3 from peer 06c63579c7234194b5f6ef457dd5e3f8 ({username='slave'} at 127.0.99.136:51407)
I20240501 07:59:57.266371 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet 4e60dad8ddd2402faae35be18662c8c3 from peer 06c63579c7234194b5f6ef457dd5e3f8 at {username='slave'} at 127.0.99.136:51407: session id = 06c63579c7234194b5f6ef457dd5e3f8-4e60dad8ddd2402faae35be18662c8c3
I20240501 07:59:57.267253 6483 tablet_copy_source_session.cc:205] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.268266 8962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e60dad8ddd2402faae35be18662c8c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.271095 8962 tablet_copy_client.cc:783] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.271378 8962 tablet_copy_client.cc:647] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.273680 8965 tablet_copy_client.cc:1002] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.282202 8960 ts_tablet_manager.cc:902] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:57.282532 8142 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 07:59:57.282927 8960 tablet_copy_client.cc:308] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 07:59:57.283434 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet 88fa1c4aaf754e32bef123157b3bc62c from peer 06c63579c7234194b5f6ef457dd5e3f8 ({username='slave'} at 127.0.99.136:47239)
I20240501 07:59:57.283542 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet 88fa1c4aaf754e32bef123157b3bc62c from peer 06c63579c7234194b5f6ef457dd5e3f8 at {username='slave'} at 127.0.99.136:47239: session id = 06c63579c7234194b5f6ef457dd5e3f8-88fa1c4aaf754e32bef123157b3bc62c
I20240501 07:59:57.284508 6352 tablet_copy_source_session.cc:205] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.285467 8960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88fa1c4aaf754e32bef123157b3bc62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.287642 8960 tablet_copy_client.cc:783] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.287793 8960 tablet_copy_client.cc:647] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.288370 8955 ts_tablet_manager.cc:902] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce: Initiating tablet copy from peer d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 07:59:57.288980 8971 tablet_copy_client.cc:1002] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.289074 8955 tablet_copy_client.cc:308] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:42501
I20240501 07:59:57.293493 6614 tablet_copy_service.cc:140] P d2e209265a41425fba98c9d9b0551bd4: Received BeginTabletCopySession request for tablet 7a9dcf4944c34707a8cd0acebc8f9029 from peer 29afadb287c049d9bb777d674b3697ce ({username='slave'} at 127.0.99.134:43243)
I20240501 07:59:57.293630 6614 tablet_copy_service.cc:161] P d2e209265a41425fba98c9d9b0551bd4: Beginning new tablet copy session on tablet 7a9dcf4944c34707a8cd0acebc8f9029 from peer 29afadb287c049d9bb777d674b3697ce at {username='slave'} at 127.0.99.134:43243: session id = 29afadb287c049d9bb777d674b3697ce-7a9dcf4944c34707a8cd0acebc8f9029
I20240501 07:59:57.294771 6614 tablet_copy_source_session.cc:205] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.296001 8955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a9dcf4944c34707a8cd0acebc8f9029. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.298682 8955 tablet_copy_client.cc:783] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.298894 8955 tablet_copy_client.cc:647] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.300447 6572 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 07:59:57.300294 8975 tablet_copy_client.cc:1002] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.312032 6181 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 07:59:57.312240 6311 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240501 07:59:57.312963 6441 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
Runtime error: ksck discovered errors
I20240501 07:59:57.356524 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.137:0
--local_ip_for_outbound_sockets=127.0.99.137
--webserver_interface=127.0.99.137
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41421
--builtin_ntp_servers=127.0.99.148:42393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 07:59:57.408988 8981 ts_tablet_manager.cc:902] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:57.409586 8981 tablet_copy_client.cc:308] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 07:59:57.410069 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet f4c62996a5a64d978850f2f7a31009f4 from peer ef83010b77b441ffac2e41468a0a1bd1 ({username='slave'} at 127.0.99.135:48397)
I20240501 07:59:57.410195 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet f4c62996a5a64d978850f2f7a31009f4 from peer ef83010b77b441ffac2e41468a0a1bd1 at {username='slave'} at 127.0.99.135:48397: session id = ef83010b77b441ffac2e41468a0a1bd1-f4c62996a5a64d978850f2f7a31009f4
I20240501 07:59:57.411144 6352 tablet_copy_source_session.cc:205] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.412791 8981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c62996a5a64d978850f2f7a31009f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.415185 8981 tablet_copy_client.cc:783] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.415416 8981 tablet_copy_client.cc:647] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.417151 8983 tablet_copy_client.cc:1002] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.448685 9012 ts_tablet_manager.cc:902] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8: Initiating tablet copy from peer 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:57.449257 9012 tablet_copy_client.cc:308] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:44189
I20240501 07:59:57.454617 6221 tablet_copy_service.cc:140] P 953b945e3e7e4c71a61aa99c1208ab1d: Received BeginTabletCopySession request for tablet 6b552a44c67940819f9ef64c68585eb1 from peer 06c63579c7234194b5f6ef457dd5e3f8 ({username='slave'} at 127.0.99.136:46025)
I20240501 07:59:57.454748 6221 tablet_copy_service.cc:161] P 953b945e3e7e4c71a61aa99c1208ab1d: Beginning new tablet copy session on tablet 6b552a44c67940819f9ef64c68585eb1 from peer 06c63579c7234194b5f6ef457dd5e3f8 at {username='slave'} at 127.0.99.136:46025: session id = 06c63579c7234194b5f6ef457dd5e3f8-6b552a44c67940819f9ef64c68585eb1
I20240501 07:59:57.455714 6221 tablet_copy_source_session.cc:205] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.456866 9012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b552a44c67940819f9ef64c68585eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.459373 9012 tablet_copy_client.cc:783] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.459522 9012 tablet_copy_client.cc:647] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.460851 9022 ts_tablet_manager.cc:902] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36: Initiating tablet copy from peer 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:57.461416 9022 tablet_copy_client.cc:308] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:44189
I20240501 07:59:57.463068 9013 tablet_copy_client.cc:1002] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.466001 6221 tablet_copy_service.cc:140] P 953b945e3e7e4c71a61aa99c1208ab1d: Received BeginTabletCopySession request for tablet 5dae6c0b9c1d4009a93ed157e1f763c8 from peer 7a80bdedf813425faf7617b4b07c9f36 ({username='slave'} at 127.0.99.133:59437)
I20240501 07:59:57.466104 6221 tablet_copy_service.cc:161] P 953b945e3e7e4c71a61aa99c1208ab1d: Beginning new tablet copy session on tablet 5dae6c0b9c1d4009a93ed157e1f763c8 from peer 7a80bdedf813425faf7617b4b07c9f36 at {username='slave'} at 127.0.99.133:59437: session id = 7a80bdedf813425faf7617b4b07c9f36-5dae6c0b9c1d4009a93ed157e1f763c8
I20240501 07:59:57.466996 6221 tablet_copy_source_session.cc:205] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.468135 9022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dae6c0b9c1d4009a93ed157e1f763c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.469857 9022 tablet_copy_client.cc:783] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.469990 9022 tablet_copy_client.cc:647] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.471045 9023 tablet_copy_client.cc:1002] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Injecting latency into file download: 1500
W20240501 07:59:57.484205 8976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 07:59:57.484452 8976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 07:59:57.484530 8976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240501 07:59:57.488180 9045 ts_tablet_manager.cc:902] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8: Initiating tablet copy from peer 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
W20240501 07:59:57.488449 8976 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 07:59:57.488588 8976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 07:59:57.488706 8976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.137
I20240501 07:59:57.488852 9045 tablet_copy_client.cc:308] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:44189
I20240501 07:59:57.489451 6221 tablet_copy_service.cc:140] P 953b945e3e7e4c71a61aa99c1208ab1d: Received BeginTabletCopySession request for tablet e952704c6a3d43aba833484bba9fddbf from peer 06c63579c7234194b5f6ef457dd5e3f8 ({username='slave'} at 127.0.99.136:46025)
I20240501 07:59:57.489589 6221 tablet_copy_service.cc:161] P 953b945e3e7e4c71a61aa99c1208ab1d: Beginning new tablet copy session on tablet e952704c6a3d43aba833484bba9fddbf from peer 06c63579c7234194b5f6ef457dd5e3f8 at {username='slave'} at 127.0.99.136:46025: session id = 06c63579c7234194b5f6ef457dd5e3f8-e952704c6a3d43aba833484bba9fddbf
I20240501 07:59:57.490415 6221 tablet_copy_source_session.cc:205] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.491416 9045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e952704c6a3d43aba833484bba9fddbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.491870 9055 ts_tablet_manager.cc:902] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36: Initiating tablet copy from peer 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:57.493374 9055 tablet_copy_client.cc:308] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:44189
I20240501 07:59:57.493465 8976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:42393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.137:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.0.99.137
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.0.99.190:41421
--never_fsync=true
--heap_profile_path=/tmp/kudu.8976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.137
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 07:59:57.493914 6221 tablet_copy_service.cc:140] P 953b945e3e7e4c71a61aa99c1208ab1d: Received BeginTabletCopySession request for tablet 5d0ae7c274af4ce1992e9507cbabc551 from peer 7a80bdedf813425faf7617b4b07c9f36 ({username='slave'} at 127.0.99.133:59437)
I20240501 07:59:57.494019 6221 tablet_copy_service.cc:161] P 953b945e3e7e4c71a61aa99c1208ab1d: Beginning new tablet copy session on tablet 5d0ae7c274af4ce1992e9507cbabc551 from peer 7a80bdedf813425faf7617b4b07c9f36 at {username='slave'} at 127.0.99.133:59437: session id = 7a80bdedf813425faf7617b4b07c9f36-5d0ae7c274af4ce1992e9507cbabc551
I20240501 07:59:57.494158 9045 tablet_copy_client.cc:783] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.494349 9045 tablet_copy_client.cc:647] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.494894 6221 tablet_copy_source_session.cc:205] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.495048 8976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 07:59:57.495424 9051 tablet_copy_client.cc:1002] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.496387 9055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d0ae7c274af4ce1992e9507cbabc551. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.496495 8976 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240501 07:59:57.498145 9055 tablet_copy_client.cc:783] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.498582 9055 tablet_copy_client.cc:647] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.499606 9059 tablet_copy_client.cc:1002] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.503846 9075 ts_tablet_manager.cc:902] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 07:59:57.505089 9075 tablet_copy_client.cc:308] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
W20240501 07:59:57.506938 9072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:57.507540 9074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 07:59:57.507994 9071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 07:59:57.508570 8976 server_base.cc:1034] running on GCE node
I20240501 07:59:57.509034 8976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 07:59:57.509390 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet 47d3021e3ac74c6ba482b8b54066c36e from peer 29afadb287c049d9bb777d674b3697ce ({username='slave'} at 127.0.99.134:36071)
I20240501 07:59:57.509496 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet 47d3021e3ac74c6ba482b8b54066c36e from peer 29afadb287c049d9bb777d674b3697ce at {username='slave'} at 127.0.99.134:36071: session id = 29afadb287c049d9bb777d674b3697ce-47d3021e3ac74c6ba482b8b54066c36e
I20240501 07:59:57.509725 8976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 07:59:57.510424 6483 tablet_copy_source_session.cc:205] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.510901 8976 hybrid_clock.cc:648] HybridClock initialized: now 1714550397510877 us; error 43 us; skew 500 ppm
I20240501 07:59:57.511243 9075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47d3021e3ac74c6ba482b8b54066c36e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.513149 8976 webserver.cc:460] Webserver started at http://127.0.99.137:43461/ using document root <none> and password file <none>
I20240501 07:59:57.513566 9075 tablet_copy_client.cc:783] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.513718 9075 tablet_copy_client.cc:647] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.513849 8976 fs_manager.cc:359] Metadata directory not provided
I20240501 07:59:57.513917 8976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 07:59:57.514132 8976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 07:59:57.516220 8976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/instance:
uuid: "773008dcb81842c3bdf9ce259bb66b8a"
format_stamp: "Formatted at 2024-05-01 07:59:57 on dist-test-slave-f81v"
I20240501 07:59:57.516819 8976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal/instance:
uuid: "773008dcb81842c3bdf9ce259bb66b8a"
format_stamp: "Formatted at 2024-05-01 07:59:57 on dist-test-slave-f81v"
I20240501 07:59:57.517741 9076 tablet_copy_client.cc:1002] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.521107 8976 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20240501 07:59:57.524132 9095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:57.525673 8976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:57.525803 8976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "773008dcb81842c3bdf9ce259bb66b8a"
format_stamp: "Formatted at 2024-05-01 07:59:57 on dist-test-slave-f81v"
I20240501 07:59:57.525915 8976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 07:59:57.558209 9120 ts_tablet_manager.cc:902] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce: Initiating tablet copy from peer 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 07:59:57.558805 9120 tablet_copy_client.cc:308] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:44189
I20240501 07:59:57.562564 6220 tablet_copy_service.cc:140] P 953b945e3e7e4c71a61aa99c1208ab1d: Received BeginTabletCopySession request for tablet d929e6e09be943b68061ad4b1bb8e9f1 from peer 29afadb287c049d9bb777d674b3697ce ({username='slave'} at 127.0.99.134:42931)
I20240501 07:59:57.562700 6220 tablet_copy_service.cc:161] P 953b945e3e7e4c71a61aa99c1208ab1d: Beginning new tablet copy session on tablet d929e6e09be943b68061ad4b1bb8e9f1 from peer 29afadb287c049d9bb777d674b3697ce at {username='slave'} at 127.0.99.134:42931: session id = 29afadb287c049d9bb777d674b3697ce-d929e6e09be943b68061ad4b1bb8e9f1
I20240501 07:59:57.563603 6220 tablet_copy_source_session.cc:205] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.564457 9120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d929e6e09be943b68061ad4b1bb8e9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.566592 9120 tablet_copy_client.cc:783] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.566742 9120 tablet_copy_client.cc:647] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.567992 9124 tablet_copy_client.cc:1002] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.568909 8976 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 07:59:57.569469 8976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 07:59:57.569622 8976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 07:59:57.570123 8976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 07:59:57.571074 8976 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 07:59:57.571127 8976 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:57.571166 8976 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 07:59:57.571187 8976 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 07:59:57.579638 8976 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.137:42389
I20240501 07:59:57.579773 9234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.137:42389 every 8 connection(s)
I20240501 07:59:57.580668 8976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240501 07:59:57.583151 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 8976
I20240501 07:59:57.583288 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal/instance
I20240501 07:59:57.590374 9235 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41421
I20240501 07:59:57.590656 9235 heartbeater.cc:464] Registering TS with master...
I20240501 07:59:57.591229 9235 heartbeater.cc:511] Master 127.0.99.190:41421 requested a full tablet report, sending...
I20240501 07:59:57.592275 6052 ts_manager.cc:194] Registered new tserver with Master: 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389)
I20240501 07:59:57.592934 6052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.137:39735
I20240501 07:59:57.614018 9238 ts_tablet_manager.cc:902] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 07:59:57.614522 9238 tablet_copy_client.cc:308] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
I20240501 07:59:57.615161 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet 947b5d16224a42dcad709e7ddefc0b33 from peer 7a80bdedf813425faf7617b4b07c9f36 ({username='slave'} at 127.0.99.133:60211)
I20240501 07:59:57.615284 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet 947b5d16224a42dcad709e7ddefc0b33 from peer 7a80bdedf813425faf7617b4b07c9f36 at {username='slave'} at 127.0.99.133:60211: session id = 7a80bdedf813425faf7617b4b07c9f36-947b5d16224a42dcad709e7ddefc0b33
I20240501 07:59:57.616149 6483 tablet_copy_source_session.cc:205] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.617023 9238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 947b5d16224a42dcad709e7ddefc0b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.618801 9238 tablet_copy_client.cc:783] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.618948 9238 tablet_copy_client.cc:647] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.619908 9242 tablet_copy_client.cc:1002] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:57.662947 9247 ts_tablet_manager.cc:902] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 07:59:57.663416 9247 tablet_copy_client.cc:308] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 07:59:57.663949 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet de48f8c2870f4517b5349c30a51fce56 from peer ef83010b77b441ffac2e41468a0a1bd1 ({username='slave'} at 127.0.99.135:48397)
I20240501 07:59:57.664052 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet de48f8c2870f4517b5349c30a51fce56 from peer ef83010b77b441ffac2e41468a0a1bd1 at {username='slave'} at 127.0.99.135:48397: session id = ef83010b77b441ffac2e41468a0a1bd1-de48f8c2870f4517b5349c30a51fce56
I20240501 07:59:57.664808 6352 tablet_copy_source_session.cc:205] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 07:59:57.665645 9247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de48f8c2870f4517b5349c30a51fce56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 07:59:57.667347 9247 tablet_copy_client.cc:783] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 0 data blocks...
I20240501 07:59:57.667486 9247 tablet_copy_client.cc:647] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 1 WAL segments...
I20240501 07:59:57.668547 9251 tablet_copy_client.cc:1002] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Injecting latency into file download: 1500
I20240501 07:59:58.553216 8765 tablet_copy_client.cc:523] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.555436 8765 tablet_bootstrap.cc:492] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap starting.
I20240501 07:59:58.561342 8765 log.cc:824] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:58.580328 8765 tablet_bootstrap.cc:492] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.580705 8765 tablet_bootstrap.cc:492] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap complete.
I20240501 07:59:58.581002 8765 ts_tablet_manager.cc:1367] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8: Time spent bootstrapping tablet: real 0.026s user 0.026s sys 0.000s
I20240501 07:59:58.583660 8765 raft_consensus.cc:348] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:58.583961 8765 raft_consensus.cc:729] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06c63579c7234194b5f6ef457dd5e3f8, State: Initialized, Role: LEARNER
I20240501 07:59:58.584332 8765 consensus_queue.cc:260] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:58.585245 8765 ts_tablet_manager.cc:1398] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 07:59:58.586282 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-3098236442b242b38c5937de465c8e1f received from {username='slave'} at 127.0.99.136:47239
I20240501 07:59:58.586380 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-3098236442b242b38c5937de465c8e1f on tablet 3098236442b242b38c5937de465c8e1f with peer 06c63579c7234194b5f6ef457dd5e3f8
I20240501 07:59:58.595160 9235 heartbeater.cc:503] Master 127.0.99.190:41421 was elected leader, sending a full tablet report...
I20240501 07:59:58.640834 8831 tablet_copy_client.cc:523] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.642802 8831 tablet_bootstrap.cc:492] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap starting.
I20240501 07:59:58.648276 8831 log.cc:824] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:58.670572 8831 tablet_bootstrap.cc:492] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.670961 8831 tablet_bootstrap.cc:492] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap complete.
I20240501 07:59:58.671275 8831 ts_tablet_manager.cc:1367] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1: Time spent bootstrapping tablet: real 0.029s user 0.027s sys 0.000s
I20240501 07:59:58.673367 8831 raft_consensus.cc:348] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:58.673642 8831 raft_consensus.cc:729] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Initialized, Role: LEARNER
I20240501 07:59:58.673964 8831 consensus_queue.cc:260] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:58.674793 8831 ts_tablet_manager.cc:1398] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
I20240501 07:59:58.675504 6220 tablet_copy_service.cc:342] P 953b945e3e7e4c71a61aa99c1208ab1d: Request end of tablet copy session ef83010b77b441ffac2e41468a0a1bd1-16752f91ea894a7ba4017170c1dc6ec1 received from {username='slave'} at 127.0.99.135:33435
I20240501 07:59:58.675652 6220 tablet_copy_service.cc:433] P 953b945e3e7e4c71a61aa99c1208ab1d: ending tablet copy session ef83010b77b441ffac2e41468a0a1bd1-16752f91ea894a7ba4017170c1dc6ec1 on tablet 16752f91ea894a7ba4017170c1dc6ec1 with peer ef83010b77b441ffac2e41468a0a1bd1
I20240501 07:59:58.695137 8881 tablet_copy_client.cc:523] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.697541 8881 tablet_bootstrap.cc:492] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce: Bootstrap starting.
I20240501 07:59:58.703563 8881 log.cc:824] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:58.705479 8898 tablet_copy_client.cc:523] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.707836 8898 tablet_bootstrap.cc:492] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap starting.
I20240501 07:59:58.714704 8898 log.cc:824] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36: Log is configured to *not* fsync() on all Append() calls
I20240501 07:59:58.728593 8881 tablet_bootstrap.cc:492] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.728989 8881 tablet_bootstrap.cc:492] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce: Bootstrap complete.
I20240501 07:59:58.729386 8881 ts_tablet_manager.cc:1367] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce: Time spent bootstrapping tablet: real 0.032s user 0.025s sys 0.005s
I20240501 07:59:58.731796 8881 raft_consensus.cc:348] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:58.732105 8881 raft_consensus.cc:729] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 29afadb287c049d9bb777d674b3697ce, State: Initialized, Role: LEARNER
I20240501 07:59:58.732466 8881 consensus_queue.cc:260] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:58.733439 8881 ts_tablet_manager.cc:1398] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 07:59:58.734185 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session 29afadb287c049d9bb777d674b3697ce-41b9f1a284c34a9099cf1014db77f35c received from {username='slave'} at 127.0.99.134:51035
I20240501 07:59:58.734274 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session 29afadb287c049d9bb777d674b3697ce-41b9f1a284c34a9099cf1014db77f35c on tablet 41b9f1a284c34a9099cf1014db77f35c with peer 29afadb287c049d9bb777d674b3697ce
I20240501 07:59:58.742348 8898 tablet_bootstrap.cc:492] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.742749 8898 tablet_bootstrap.cc:492] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap complete.
I20240501 07:59:58.743105 8898 ts_tablet_manager.cc:1367] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36: Time spent bootstrapping tablet: real 0.035s user 0.033s sys 0.000s
I20240501 07:59:58.745697 8898 raft_consensus.cc:348] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:58.746003 8898 raft_consensus.cc:729] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a80bdedf813425faf7617b4b07c9f36, State: Initialized, Role: LEARNER
I20240501 07:59:58.746413 8898 consensus_queue.cc:260] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:58.747399 8898 ts_tablet_manager.cc:1398] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 07:59:58.748349 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session 7a80bdedf813425faf7617b4b07c9f36-6f24f629734649a3b31c7e293122d7ce received from {username='slave'} at 127.0.99.133:60211
I20240501 07:59:58.748469 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session 7a80bdedf813425faf7617b4b07c9f36-6f24f629734649a3b31c7e293122d7ce on tablet 6f24f629734649a3b31c7e293122d7ce with peer 7a80bdedf813425faf7617b4b07c9f36
I20240501 07:59:58.754174 8939 tablet_copy_client.cc:523] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.755802 8939 tablet_bootstrap.cc:492] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap starting.
I20240501 07:59:58.757263 8934 tablet_copy_client.cc:523] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.759608 8934 tablet_bootstrap.cc:492] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap starting.
I20240501 07:59:58.762856 8956 tablet_copy_client.cc:523] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.764364 8956 tablet_bootstrap.cc:492] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap starting.
I20240501 07:59:58.774011 8962 tablet_copy_client.cc:523] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.775640 8962 tablet_bootstrap.cc:492] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap starting.
I20240501 07:59:58.787158 8934 tablet_bootstrap.cc:492] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.787577 8934 tablet_bootstrap.cc:492] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap complete.
I20240501 07:59:58.787835 8934 ts_tablet_manager.cc:1367] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36: Time spent bootstrapping tablet: real 0.028s user 0.026s sys 0.000s
I20240501 07:59:58.788095 8939 tablet_bootstrap.cc:492] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.788458 8939 tablet_bootstrap.cc:492] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap complete.
I20240501 07:59:58.788533 8934 raft_consensus.cc:348] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:58.788662 8939 ts_tablet_manager.cc:1367] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1: Time spent bootstrapping tablet: real 0.033s user 0.026s sys 0.000s
I20240501 07:59:58.788751 8934 raft_consensus.cc:729] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a80bdedf813425faf7617b4b07c9f36, State: Initialized, Role: LEARNER
I20240501 07:59:58.788903 8934 consensus_queue.cc:260] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:58.789247 8939 raft_consensus.cc:348] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:58.789394 8960 tablet_copy_client.cc:523] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.789422 8939 raft_consensus.cc:729] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Initialized, Role: LEARNER
I20240501 07:59:58.789525 8939 consensus_queue.cc:260] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:58.789839 8934 ts_tablet_manager.cc:1398] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 07:59:58.791071 8960 tablet_bootstrap.cc:492] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap starting.
I20240501 07:59:58.791369 8939 ts_tablet_manager.cc:1398] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:58.794152 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session ef83010b77b441ffac2e41468a0a1bd1-8bf587ac1d8141e99fceab28d89e4ea9 received from {username='slave'} at 127.0.99.135:48397
I20240501 07:59:58.794239 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session ef83010b77b441ffac2e41468a0a1bd1-8bf587ac1d8141e99fceab28d89e4ea9 on tablet 8bf587ac1d8141e99fceab28d89e4ea9 with peer ef83010b77b441ffac2e41468a0a1bd1
I20240501 07:59:58.794377 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session 7a80bdedf813425faf7617b4b07c9f36-d2a4fd236f394ba0ad1812c50b7e66a3 received from {username='slave'} at 127.0.99.133:60211
I20240501 07:59:58.794456 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session 7a80bdedf813425faf7617b4b07c9f36-d2a4fd236f394ba0ad1812c50b7e66a3 on tablet d2a4fd236f394ba0ad1812c50b7e66a3 with peer 7a80bdedf813425faf7617b4b07c9f36
I20240501 07:59:58.795657 8956 tablet_bootstrap.cc:492] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.795903 8956 tablet_bootstrap.cc:492] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap complete.
I20240501 07:59:58.796175 8956 ts_tablet_manager.cc:1367] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1: Time spent bootstrapping tablet: real 0.032s user 0.025s sys 0.004s
I20240501 07:59:58.796624 8956 raft_consensus.cc:348] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:58.796756 8956 raft_consensus.cc:729] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Initialized, Role: LEARNER
I20240501 07:59:58.796890 8956 consensus_queue.cc:260] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:58.797156 8956 ts_tablet_manager.cc:1398] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:58.797255 8162 raft_consensus.cc:1203] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:58.797771 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session ef83010b77b441ffac2e41468a0a1bd1-5a31c7d3de834d1aab1ffa16e6df7e15 received from {username='slave'} at 127.0.99.135:48397
I20240501 07:59:58.797855 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session ef83010b77b441ffac2e41468a0a1bd1-5a31c7d3de834d1aab1ffa16e6df7e15 on tablet 5a31c7d3de834d1aab1ffa16e6df7e15 with peer ef83010b77b441ffac2e41468a0a1bd1
I20240501 07:59:58.808761 8955 tablet_copy_client.cc:523] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.810076 8955 tablet_bootstrap.cc:492] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce: Bootstrap starting.
I20240501 07:59:58.812316 8162 raft_consensus.cc:1203] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:58.819051 8960 tablet_bootstrap.cc:492] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.819416 8960 tablet_bootstrap.cc:492] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap complete.
I20240501 07:59:58.819622 8960 ts_tablet_manager.cc:1367] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8: Time spent bootstrapping tablet: real 0.029s user 0.014s sys 0.005s
I20240501 07:59:58.820307 8960 raft_consensus.cc:348] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:58.820454 8960 raft_consensus.cc:729] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06c63579c7234194b5f6ef457dd5e3f8, State: Initialized, Role: LEARNER
I20240501 07:59:58.820602 8960 consensus_queue.cc:260] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:58.820899 8960 ts_tablet_manager.cc:1398] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240501 07:59:58.821662 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-88fa1c4aaf754e32bef123157b3bc62c received from {username='slave'} at 127.0.99.136:47239
I20240501 07:59:58.821851 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-88fa1c4aaf754e32bef123157b3bc62c on tablet 88fa1c4aaf754e32bef123157b3bc62c with peer 06c63579c7234194b5f6ef457dd5e3f8
I20240501 07:59:58.829100 8962 tablet_bootstrap.cc:492] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.829428 8962 tablet_bootstrap.cc:492] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap complete.
I20240501 07:59:58.829628 8962 ts_tablet_manager.cc:1367] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8: Time spent bootstrapping tablet: real 0.054s user 0.026s sys 0.000s
I20240501 07:59:58.830113 8962 raft_consensus.cc:348] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:58.830247 8962 raft_consensus.cc:729] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06c63579c7234194b5f6ef457dd5e3f8, State: Initialized, Role: LEARNER
I20240501 07:59:58.830348 8962 consensus_queue.cc:260] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:58.831451 8962 ts_tablet_manager.cc:1398] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240501 07:59:58.832298 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-4e60dad8ddd2402faae35be18662c8c3 received from {username='slave'} at 127.0.99.136:51407
I20240501 07:59:58.832392 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-4e60dad8ddd2402faae35be18662c8c3 on tablet 4e60dad8ddd2402faae35be18662c8c3 with peer 06c63579c7234194b5f6ef457dd5e3f8
I20240501 07:59:58.838339 8955 tablet_bootstrap.cc:492] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.838608 8955 tablet_bootstrap.cc:492] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce: Bootstrap complete.
I20240501 07:59:58.838795 8955 ts_tablet_manager.cc:1367] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce: Time spent bootstrapping tablet: real 0.029s user 0.018s sys 0.004s
I20240501 07:59:58.839433 8955 raft_consensus.cc:348] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:58.839556 8955 raft_consensus.cc:729] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 29afadb287c049d9bb777d674b3697ce, State: Initialized, Role: LEARNER
I20240501 07:59:58.839651 8955 consensus_queue.cc:260] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:58.839972 8955 ts_tablet_manager.cc:1398] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce: Time spent starting tablet: real 0.001s user 0.003s sys 0.001s
I20240501 07:59:58.840574 6614 tablet_copy_service.cc:342] P d2e209265a41425fba98c9d9b0551bd4: Request end of tablet copy session 29afadb287c049d9bb777d674b3697ce-7a9dcf4944c34707a8cd0acebc8f9029 received from {username='slave'} at 127.0.99.134:43243
I20240501 07:59:58.840663 6614 tablet_copy_service.cc:433] P d2e209265a41425fba98c9d9b0551bd4: ending tablet copy session 29afadb287c049d9bb777d674b3697ce-7a9dcf4944c34707a8cd0acebc8f9029 on tablet 7a9dcf4944c34707a8cd0acebc8f9029 with peer 29afadb287c049d9bb777d674b3697ce
I20240501 07:59:58.845211 8162 raft_consensus.cc:1203] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:58.886584 8294 raft_consensus.cc:1203] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:58.917507 8981 tablet_copy_client.cc:523] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.918990 8981 tablet_bootstrap.cc:492] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap starting.
I20240501 07:59:58.927855 7900 raft_consensus.cc:1203] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:58.938933 8981 tablet_bootstrap.cc:492] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.939245 8981 tablet_bootstrap.cc:492] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap complete.
I20240501 07:59:58.939415 8981 ts_tablet_manager.cc:1367] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1: Time spent bootstrapping tablet: real 0.020s user 0.013s sys 0.005s
I20240501 07:59:58.939860 8981 raft_consensus.cc:348] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:58.940047 8981 raft_consensus.cc:729] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Initialized, Role: LEARNER
I20240501 07:59:58.940168 8981 consensus_queue.cc:260] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:58.940917 8981 ts_tablet_manager.cc:1398] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 07:59:58.941506 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session ef83010b77b441ffac2e41468a0a1bd1-f4c62996a5a64d978850f2f7a31009f4 received from {username='slave'} at 127.0.99.135:48397
I20240501 07:59:58.941592 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session ef83010b77b441ffac2e41468a0a1bd1-f4c62996a5a64d978850f2f7a31009f4 on tablet f4c62996a5a64d978850f2f7a31009f4 with peer ef83010b77b441ffac2e41468a0a1bd1
I20240501 07:59:58.963379 9012 tablet_copy_client.cc:523] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.964797 9012 tablet_bootstrap.cc:492] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap starting.
I20240501 07:59:58.971365 9022 tablet_copy_client.cc:523] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.972846 9022 tablet_bootstrap.cc:492] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap starting.
I20240501 07:59:58.983594 9012 tablet_bootstrap.cc:492] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.983918 9012 tablet_bootstrap.cc:492] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap complete.
I20240501 07:59:58.984140 9012 ts_tablet_manager.cc:1367] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8: Time spent bootstrapping tablet: real 0.019s user 0.020s sys 0.000s
I20240501 07:59:58.984583 9012 raft_consensus.cc:348] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:58.984699 9012 raft_consensus.cc:729] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06c63579c7234194b5f6ef457dd5e3f8, State: Initialized, Role: LEARNER
I20240501 07:59:58.984795 9012 consensus_queue.cc:260] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:58.985056 9012 ts_tablet_manager.cc:1398] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:58.985672 6220 tablet_copy_service.cc:342] P 953b945e3e7e4c71a61aa99c1208ab1d: Request end of tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-6b552a44c67940819f9ef64c68585eb1 received from {username='slave'} at 127.0.99.136:46025
I20240501 07:59:58.985760 6220 tablet_copy_service.cc:433] P 953b945e3e7e4c71a61aa99c1208ab1d: ending tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-6b552a44c67940819f9ef64c68585eb1 on tablet 6b552a44c67940819f9ef64c68585eb1 with peer 06c63579c7234194b5f6ef457dd5e3f8
I20240501 07:59:58.992247 9022 tablet_bootstrap.cc:492] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:58.992555 9022 tablet_bootstrap.cc:492] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap complete.
I20240501 07:59:58.992741 9022 ts_tablet_manager.cc:1367] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36: Time spent bootstrapping tablet: real 0.020s user 0.020s sys 0.000s
I20240501 07:59:58.993180 9022 raft_consensus.cc:348] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:58.993296 9022 raft_consensus.cc:729] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a80bdedf813425faf7617b4b07c9f36, State: Initialized, Role: LEARNER
I20240501 07:59:58.993379 9022 consensus_queue.cc:260] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:58.993659 9022 ts_tablet_manager.cc:1398] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:58.994324 6220 tablet_copy_service.cc:342] P 953b945e3e7e4c71a61aa99c1208ab1d: Request end of tablet copy session 7a80bdedf813425faf7617b4b07c9f36-5dae6c0b9c1d4009a93ed157e1f763c8 received from {username='slave'} at 127.0.99.133:59437
I20240501 07:59:58.994426 6220 tablet_copy_service.cc:433] P 953b945e3e7e4c71a61aa99c1208ab1d: ending tablet copy session 7a80bdedf813425faf7617b4b07c9f36-5dae6c0b9c1d4009a93ed157e1f763c8 on tablet 5dae6c0b9c1d4009a93ed157e1f763c8 with peer 7a80bdedf813425faf7617b4b07c9f36
I20240501 07:59:58.995782 9045 tablet_copy_client.cc:523] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:58.997316 9045 tablet_bootstrap.cc:492] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap starting.
I20240501 07:59:58.999984 9055 tablet_copy_client.cc:523] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:59.001427 9055 tablet_bootstrap.cc:492] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap starting.
I20240501 07:59:59.007372 7900 raft_consensus.cc:1203] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.018100 9075 tablet_copy_client.cc:523] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:59.019506 9075 tablet_bootstrap.cc:492] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce: Bootstrap starting.
I20240501 07:59:59.023919 9045 tablet_bootstrap.cc:492] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:59.024307 9045 tablet_bootstrap.cc:492] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap complete.
I20240501 07:59:59.024530 9045 ts_tablet_manager.cc:1367] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8: Time spent bootstrapping tablet: real 0.027s user 0.025s sys 0.002s
I20240501 07:59:59.024662 8031 raft_consensus.cc:1203] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.025481 9045 raft_consensus.cc:348] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:59.025658 9045 raft_consensus.cc:729] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06c63579c7234194b5f6ef457dd5e3f8, State: Initialized, Role: LEARNER
I20240501 07:59:59.025797 9045 consensus_queue.cc:260] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:59.026212 9045 ts_tablet_manager.cc:1398] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 07:59:59.026657 6220 tablet_copy_service.cc:342] P 953b945e3e7e4c71a61aa99c1208ab1d: Request end of tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-e952704c6a3d43aba833484bba9fddbf received from {username='slave'} at 127.0.99.136:46025
I20240501 07:59:59.026738 6220 tablet_copy_service.cc:433] P 953b945e3e7e4c71a61aa99c1208ab1d: ending tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-e952704c6a3d43aba833484bba9fddbf on tablet e952704c6a3d43aba833484bba9fddbf with peer 06c63579c7234194b5f6ef457dd5e3f8
I20240501 07:59:59.028736 9055 tablet_bootstrap.cc:492] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:59.029048 9055 tablet_bootstrap.cc:492] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap complete.
I20240501 07:59:59.029217 9055 ts_tablet_manager.cc:1367] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36: Time spent bootstrapping tablet: real 0.028s user 0.026s sys 0.000s
I20240501 07:59:59.029611 9055 raft_consensus.cc:348] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:59.029734 9055 raft_consensus.cc:729] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a80bdedf813425faf7617b4b07c9f36, State: Initialized, Role: LEARNER
I20240501 07:59:59.029863 9055 consensus_queue.cc:260] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:59.030175 9055 ts_tablet_manager.cc:1398] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 07:59:59.030395 8294 raft_consensus.cc:1203] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.030771 6220 tablet_copy_service.cc:342] P 953b945e3e7e4c71a61aa99c1208ab1d: Request end of tablet copy session 7a80bdedf813425faf7617b4b07c9f36-5d0ae7c274af4ce1992e9507cbabc551 received from {username='slave'} at 127.0.99.133:59437
I20240501 07:59:59.030859 6220 tablet_copy_service.cc:433] P 953b945e3e7e4c71a61aa99c1208ab1d: ending tablet copy session 7a80bdedf813425faf7617b4b07c9f36-5d0ae7c274af4ce1992e9507cbabc551 on tablet 5d0ae7c274af4ce1992e9507cbabc551 with peer 7a80bdedf813425faf7617b4b07c9f36
I20240501 07:59:59.039549 7900 raft_consensus.cc:1203] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.043843 9075 tablet_bootstrap.cc:492] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:59.044188 9075 tablet_bootstrap.cc:492] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce: Bootstrap complete.
I20240501 07:59:59.044373 9075 ts_tablet_manager.cc:1367] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce: Time spent bootstrapping tablet: real 0.025s user 0.022s sys 0.001s
I20240501 07:59:59.044757 9075 raft_consensus.cc:348] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:59.044863 9075 raft_consensus.cc:729] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 29afadb287c049d9bb777d674b3697ce, State: Initialized, Role: LEARNER
I20240501 07:59:59.044936 9075 consensus_queue.cc:260] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:59.045204 9075 ts_tablet_manager.cc:1398] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 07:59:59.045765 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session 29afadb287c049d9bb777d674b3697ce-47d3021e3ac74c6ba482b8b54066c36e received from {username='slave'} at 127.0.99.134:36071
I20240501 07:59:59.045840 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session 29afadb287c049d9bb777d674b3697ce-47d3021e3ac74c6ba482b8b54066c36e on tablet 47d3021e3ac74c6ba482b8b54066c36e with peer 29afadb287c049d9bb777d674b3697ce
I20240501 07:59:59.059944 8294 raft_consensus.cc:1203] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.068305 9120 tablet_copy_client.cc:523] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:59.069900 9120 tablet_bootstrap.cc:492] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce: Bootstrap starting.
I20240501 07:59:59.090283 9120 tablet_bootstrap.cc:492] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:59.090596 9120 tablet_bootstrap.cc:492] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce: Bootstrap complete.
I20240501 07:59:59.090786 9120 ts_tablet_manager.cc:1367] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce: Time spent bootstrapping tablet: real 0.021s user 0.019s sys 0.000s
I20240501 07:59:59.091179 9120 raft_consensus.cc:348] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:59.091306 9120 raft_consensus.cc:729] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 29afadb287c049d9bb777d674b3697ce, State: Initialized, Role: LEARNER
I20240501 07:59:59.091416 9120 consensus_queue.cc:260] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:59.091845 9120 ts_tablet_manager.cc:1398] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 07:59:59.092346 6220 tablet_copy_service.cc:342] P 953b945e3e7e4c71a61aa99c1208ab1d: Request end of tablet copy session 29afadb287c049d9bb777d674b3697ce-d929e6e09be943b68061ad4b1bb8e9f1 received from {username='slave'} at 127.0.99.134:42931
I20240501 07:59:59.092437 6220 tablet_copy_service.cc:433] P 953b945e3e7e4c71a61aa99c1208ab1d: ending tablet copy session 29afadb287c049d9bb777d674b3697ce-d929e6e09be943b68061ad4b1bb8e9f1 on tablet d929e6e09be943b68061ad4b1bb8e9f1 with peer 29afadb287c049d9bb777d674b3697ce
I20240501 07:59:59.120247 9238 tablet_copy_client.cc:523] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:59.121635 9238 tablet_bootstrap.cc:492] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap starting.
I20240501 07:59:59.141993 9238 tablet_bootstrap.cc:492] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:59.142661 9238 tablet_bootstrap.cc:492] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap complete.
I20240501 07:59:59.142925 9238 ts_tablet_manager.cc:1367] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36: Time spent bootstrapping tablet: real 0.021s user 0.022s sys 0.000s
I20240501 07:59:59.143481 9238 raft_consensus.cc:348] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:59.143698 9238 raft_consensus.cc:729] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a80bdedf813425faf7617b4b07c9f36, State: Initialized, Role: LEARNER
I20240501 07:59:59.143852 9238 consensus_queue.cc:260] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:59.146164 9238 ts_tablet_manager.cc:1398] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 07:59:59.148620 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session 7a80bdedf813425faf7617b4b07c9f36-947b5d16224a42dcad709e7ddefc0b33 received from {username='slave'} at 127.0.99.133:60211
I20240501 07:59:59.148711 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session 7a80bdedf813425faf7617b4b07c9f36-947b5d16224a42dcad709e7ddefc0b33 on tablet 947b5d16224a42dcad709e7ddefc0b33 with peer 7a80bdedf813425faf7617b4b07c9f36
I20240501 07:59:59.153044 8031 raft_consensus.cc:1203] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.168872 9247 tablet_copy_client.cc:523] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 07:59:59.170236 9247 tablet_bootstrap.cc:492] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap starting.
I20240501 07:59:59.193961 9247 tablet_bootstrap.cc:492] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 07:59:59.194408 9247 tablet_bootstrap.cc:492] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap complete.
I20240501 07:59:59.194608 9247 ts_tablet_manager.cc:1367] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1: Time spent bootstrapping tablet: real 0.024s user 0.019s sys 0.004s
I20240501 07:59:59.195076 9247 raft_consensus.cc:348] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:59.195191 9247 raft_consensus.cc:729] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Initialized, Role: LEARNER
I20240501 07:59:59.195482 9247 consensus_queue.cc:260] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:59.195876 9247 ts_tablet_manager.cc:1398] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 07:59:59.196439 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session ef83010b77b441ffac2e41468a0a1bd1-de48f8c2870f4517b5349c30a51fce56 received from {username='slave'} at 127.0.99.135:48397
I20240501 07:59:59.196533 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session ef83010b77b441ffac2e41468a0a1bd1-de48f8c2870f4517b5349c30a51fce56 on tablet de48f8c2870f4517b5349c30a51fce56 with peer ef83010b77b441ffac2e41468a0a1bd1
I20240501 07:59:59.220925 8294 raft_consensus.cc:1203] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.230727 7333 raft_consensus.cc:1050] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: attempting to promote NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 to VOTER
I20240501 07:59:59.231356 7333 consensus_queue.cc:237] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 07:59:59.232966 6593 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.233060 6463 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.233407 9754 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.233470 8162 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.233650 7333 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.233927 8607 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.237103 9754 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.237761 6593 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.238080 8162 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.239770 6463 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.240540 6052 catalog_manager.cc:5484] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.244732 6201 consensus_queue.cc:237] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 07:59:59.245776 6463 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.245800 8162 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.246142 8607 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.246327 7333 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.248247 8607 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.249701 8162 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.249828 6463 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.250789 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 16752f91ea894a7ba4017170c1dc6ec1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.252157 6054 catalog_manager.cc:5484] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.254611 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet 16752f91ea894a7ba4017170c1dc6ec1 with delete_type TABLET_DATA_TOMBSTONED (TS d2e209265a41425fba98c9d9b0551bd4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60522
I20240501 07:59:59.255148 9762 tablet_replica.cc:330] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4: stopping tablet replica
I20240501 07:59:59.255517 9762 raft_consensus.cc:2229] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.255738 9762 raft_consensus.cc:2258] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.256304 9763 raft_consensus.cc:1050] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 to VOTER
I20240501 07:59:59.256795 9763 consensus_queue.cc:237] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 07:59:59.257211 9762 ts_tablet_manager.cc:1875] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.258231 8162 raft_consensus.cc:1261] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.258633 8400 consensus_queue.cc:1035] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.258857 6201 raft_consensus.cc:1261] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.259299 8400 consensus_queue.cc:1035] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.259539 6593 raft_consensus.cc:1261] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.259686 9762 ts_tablet_manager.cc:1888] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.259861 9762 log.cc:1196] T 16752f91ea894a7ba4017170c1dc6ec1 P d2e209265a41425fba98c9d9b0551bd4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/16752f91ea894a7ba4017170c1dc6ec1
I20240501 07:59:59.259917 8400 consensus_queue.cc:1035] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.260649 6046 catalog_manager.cc:4830] TS d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501): tablet 16752f91ea894a7ba4017170c1dc6ec1 (table rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]) successfully deleted
I20240501 07:59:59.261308 7817 raft_consensus.cc:2932] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.261759 8162 raft_consensus.cc:2932] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.262773 6593 raft_consensus.cc:2932] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.264231 6057 catalog_manager.cc:5484] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.264883 6201 raft_consensus.cc:2932] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.267603 6329 consensus_queue.cc:237] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 07:59:59.268910 8162 raft_consensus.cc:1261] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.268966 6593 raft_consensus.cc:1261] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.269418 9763 consensus_queue.cc:1035] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.269642 8400 consensus_queue.cc:1035] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.271428 7817 raft_consensus.cc:2932] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.271852 6593 raft_consensus.cc:2932] T 5a31c7d3de834d1aab1ffa16e6df7e15 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.271867 8162 raft_consensus.cc:2932] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.273170 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5a31c7d3de834d1aab1ffa16e6df7e15 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.274593 6054 catalog_manager.cc:5484] T 5a31c7d3de834d1aab1ffa16e6df7e15 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.277323 6181 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a31c7d3de834d1aab1ffa16e6df7e15 with delete_type TABLET_DATA_TOMBSTONED (TS 953b945e3e7e4c71a61aa99c1208ab1d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45448
I20240501 07:59:59.277864 9768 tablet_replica.cc:330] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d: stopping tablet replica
I20240501 07:59:59.278159 9768 raft_consensus.cc:2229] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.278371 9768 raft_consensus.cc:2258] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.279477 9768 ts_tablet_manager.cc:1875] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.281502 9768 ts_tablet_manager.cc:1888] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.281587 9768 log.cc:1196] T 5a31c7d3de834d1aab1ffa16e6df7e15 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/5a31c7d3de834d1aab1ffa16e6df7e15
I20240501 07:59:59.282166 6046 catalog_manager.cc:4830] TS 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189): tablet 5a31c7d3de834d1aab1ffa16e6df7e15 (table rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]) successfully deleted
I20240501 07:59:59.305168 8294 raft_consensus.cc:1203] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.305837 9763 raft_consensus.cc:1050] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 to VOTER
I20240501 07:59:59.306294 9763 consensus_queue.cc:237] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 07:59:59.307616 8162 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.307679 6463 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.307711 6593 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.307960 9769 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.308156 8400 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.308368 9763 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.310185 8400 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.310671 6463 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.310719 6593 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.312271 8162 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.313321 6051 catalog_manager.cc:5484] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.316394 6329 consensus_queue.cc:237] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 07:59:59.317512 8162 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.317528 6593 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.317965 9769 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.318166 8400 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.319797 9763 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.320312 6593 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.320314 8162 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.321493 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8bf587ac1d8141e99fceab28d89e4ea9 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.322645 6057 catalog_manager.cc:5484] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 07:59:59.325048 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bf587ac1d8141e99fceab28d89e4ea9 with delete_type TABLET_DATA_TOMBSTONED (TS 5db2970f4bbd4a3c92d7d913f99716b1 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:51996
I20240501 07:59:59.325660 9774 tablet_replica.cc:330] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 07:59:59.326239 9774 raft_consensus.cc:2229] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.326283 7900 raft_consensus.cc:1203] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.326534 9774 raft_consensus.cc:2258] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.327713 9774 ts_tablet_manager.cc:1875] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.329571 9774 ts_tablet_manager.cc:1888] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.329655 9774 log.cc:1196] T 8bf587ac1d8141e99fceab28d89e4ea9 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/8bf587ac1d8141e99fceab28d89e4ea9
I20240501 07:59:59.330174 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet 8bf587ac1d8141e99fceab28d89e4ea9 (table rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]) successfully deleted
I20240501 07:59:59.357774 8031 raft_consensus.cc:1203] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.393371 8400 raft_consensus.cc:1050] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 to VOTER
I20240501 07:59:59.393855 8400 consensus_queue.cc:237] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 07:59:59.395303 6198 raft_consensus.cc:1261] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.395295 6463 raft_consensus.cc:1261] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.395435 8294 raft_consensus.cc:1261] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.395802 9769 consensus_queue.cc:1035] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.396035 8400 consensus_queue.cc:1035] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.396209 9763 consensus_queue.cc:1035] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.399452 8400 raft_consensus.cc:2932] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.400166 8294 raft_consensus.cc:2932] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.402256 6463 raft_consensus.cc:2932] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.403946 6052 catalog_manager.cc:5484] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8 reported cstate change: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.405826 6198 raft_consensus.cc:2932] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.409351 6329 consensus_queue.cc:237] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 07:59:59.410562 6463 raft_consensus.cc:1261] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.410629 8294 raft_consensus.cc:1261] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.410980 7817 consensus_queue.cc:1035] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.411334 8400 consensus_queue.cc:1035] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.413002 7817 raft_consensus.cc:2932] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.413738 6463 raft_consensus.cc:2932] T 3098236442b242b38c5937de465c8e1f P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.414283 8294 raft_consensus.cc:2932] T 3098236442b242b38c5937de465c8e1f P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.414661 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3098236442b242b38c5937de465c8e1f with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.415886 6057 catalog_manager.cc:5484] T 3098236442b242b38c5937de465c8e1f P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.418710 6181 tablet_service.cc:1514] Processing DeleteTablet for tablet 3098236442b242b38c5937de465c8e1f with delete_type TABLET_DATA_TOMBSTONED (TS 953b945e3e7e4c71a61aa99c1208ab1d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45448
I20240501 07:59:59.419314 9768 tablet_replica.cc:330] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d: stopping tablet replica
I20240501 07:59:59.419525 9768 raft_consensus.cc:2229] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.419700 9768 raft_consensus.cc:2258] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.420483 9768 ts_tablet_manager.cc:1875] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.422343 9768 ts_tablet_manager.cc:1888] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.422408 9768 log.cc:1196] T 3098236442b242b38c5937de465c8e1f P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/3098236442b242b38c5937de465c8e1f
I20240501 07:59:59.422940 6046 catalog_manager.cc:4830] TS 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189): tablet 3098236442b242b38c5937de465c8e1f (table rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]) successfully deleted
I20240501 07:59:59.433091 9794 raft_consensus.cc:1050] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 to VOTER
I20240501 07:59:59.433633 9794 consensus_queue.cc:237] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 07:59:59.435149 7900 raft_consensus.cc:1261] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.435287 6329 raft_consensus.cc:1261] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.435639 6198 raft_consensus.cc:1261] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.435735 9794 consensus_queue.cc:1035] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.436103 9800 consensus_queue.cc:1035] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.436314 9794 consensus_queue.cc:1035] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.438757 9800 raft_consensus.cc:2932] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.439568 6329 raft_consensus.cc:2932] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.440846 6198 raft_consensus.cc:2932] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.441870 6054 catalog_manager.cc:5484] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.443040 9246 raft_consensus.cc:1050] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 to VOTER
I20240501 07:59:59.443442 9246 consensus_queue.cc:237] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 07:59:59.445770 6593 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.446456 6329 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.446996 6463 consensus_queue.cc:237] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 07:59:59.448345 7898 raft_consensus.cc:1261] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.448500 8617 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.448772 7899 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.448915 9794 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.449362 7900 raft_consensus.cc:2932] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.449931 9794 consensus_queue.cc:1035] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.450229 9794 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.451385 6198 raft_consensus.cc:1261] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.451786 9806 consensus_queue.cc:1035] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.453310 9805 raft_consensus.cc:2932] T d2a4fd236f394ba0ad1812c50b7e66a3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.453573 9794 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.453848 7899 raft_consensus.cc:2932] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.455328 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d2a4fd236f394ba0ad1812c50b7e66a3 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.455641 6593 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.456099 6329 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.456323 6052 catalog_manager.cc:5484] T d2a4fd236f394ba0ad1812c50b7e66a3 P 7a80bdedf813425faf7617b4b07c9f36 reported cstate change: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.456470 7899 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.459664 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet d2a4fd236f394ba0ad1812c50b7e66a3 with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60224
I20240501 07:59:59.460223 6057 catalog_manager.cc:5484] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.460615 9811 tablet_replica.cc:330] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 07:59:59.460855 9811 raft_consensus.cc:2229] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.460997 9811 raft_consensus.cc:2258] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.463735 8162 raft_consensus.cc:1203] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.464069 6463 consensus_queue.cc:237] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 07:59:59.464679 9811 ts_tablet_manager.cc:1875] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.465389 6593 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.466542 6198 raft_consensus.cc:2932] T d2a4fd236f394ba0ad1812c50b7e66a3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.466907 8617 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.467197 7900 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.467610 9811 ts_tablet_manager.cc:1888] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.467644 8617 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.467706 9811 log.cc:1196] T d2a4fd236f394ba0ad1812c50b7e66a3 P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/d2a4fd236f394ba0ad1812c50b7e66a3
I20240501 07:59:59.468932 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet d2a4fd236f394ba0ad1812c50b7e66a3 (table rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]) successfully deleted
I20240501 07:59:59.470140 8617 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.472026 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6f24f629734649a3b31c7e293122d7ce with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.472307 6593 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.472877 7900 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.474483 8031 raft_consensus.cc:1203] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.474679 6051 catalog_manager.cc:5484] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.477689 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f24f629734649a3b31c7e293122d7ce with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60224
I20240501 07:59:59.477911 9811 tablet_replica.cc:330] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 07:59:59.478027 9811 raft_consensus.cc:2229] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.478166 9811 raft_consensus.cc:2258] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.478866 9811 ts_tablet_manager.cc:1875] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.480609 9811 ts_tablet_manager.cc:1888] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.480700 9811 log.cc:1196] T 6f24f629734649a3b31c7e293122d7ce P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/6f24f629734649a3b31c7e293122d7ce
I20240501 07:59:59.481369 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet 6f24f629734649a3b31c7e293122d7ce (table rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]) successfully deleted
I20240501 07:59:59.505385 7900 raft_consensus.cc:1203] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.544870 8162 raft_consensus.cc:1203] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 07:59:59.551268 8400 raft_consensus.cc:1050] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER 29afadb287c049d9bb777d674b3697ce to VOTER
I20240501 07:59:59.551674 8400 consensus_queue.cc:237] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 07:59:59.553052 6593 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.553058 6463 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.553419 8400 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.553714 9769 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.553820 8031 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.555040 9769 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.556538 8400 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.557993 6463 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.560070 9754 raft_consensus.cc:1050] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d: attempting to promote NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 to VOTER
I20240501 07:59:59.560474 9754 consensus_queue.cc:237] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 07:59:59.561209 8031 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.561980 8294 raft_consensus.cc:1261] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.562237 6329 raft_consensus.cc:1261] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.562341 7333 consensus_queue.cc:1035] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.562671 7333 consensus_queue.cc:1035] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.564275 6057 catalog_manager.cc:5484] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce reported cstate change: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.564934 6463 raft_consensus.cc:1261] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.565342 9752 consensus_queue.cc:1035] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.566267 6593 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.567308 7333 raft_consensus.cc:2932] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.567799 6329 raft_consensus.cc:2932] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.569459 8294 raft_consensus.cc:2932] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.569846 6054 catalog_manager.cc:5484] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.569922 6463 raft_consensus.cc:2932] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.573778 6198 consensus_queue.cc:237] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 07:59:59.575387 8294 raft_consensus.cc:1261] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.575574 6463 raft_consensus.cc:1261] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.575822 8607 consensus_queue.cc:1035] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.576050 9754 consensus_queue.cc:1035] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.576833 6329 consensus_queue.cc:237] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 07:59:59.578038 6462 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.578454 7817 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.578693 8031 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.579057 7817 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.579741 8607 raft_consensus.cc:2932] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.580121 7817 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.581655 6462 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.582466 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e952704c6a3d43aba833484bba9fddbf with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.582674 6054 catalog_manager.cc:5484] T e952704c6a3d43aba833484bba9fddbf P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.583042 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 41b9f1a284c34a9099cf1014db77f35c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.584467 6055 catalog_manager.cc:5484] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:59.585175 6463 raft_consensus.cc:2932] T e952704c6a3d43aba833484bba9fddbf P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.586946 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet e952704c6a3d43aba833484bba9fddbf with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60224
I20240501 07:59:59.586948 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b9f1a284c34a9099cf1014db77f35c with delete_type TABLET_DATA_TOMBSTONED (TS d2e209265a41425fba98c9d9b0551bd4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60522
I20240501 07:59:59.587163 9762 tablet_replica.cc:330] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4: stopping tablet replica
I20240501 07:59:59.587289 9762 raft_consensus.cc:2229] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.587358 9811 tablet_replica.cc:330] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 07:59:59.587206 8294 raft_consensus.cc:2932] T e952704c6a3d43aba833484bba9fddbf P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.587451 9762 raft_consensus.cc:2258] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.587477 9811 raft_consensus.cc:2229] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.587603 9811 raft_consensus.cc:2258] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.588194 9762 ts_tablet_manager.cc:1875] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.588410 9811 ts_tablet_manager.cc:1875] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.589793 9752 raft_consensus.cc:1050] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d: attempting to promote NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 to VOTER
I20240501 07:59:59.590231 9752 consensus_queue.cc:237] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 07:59:59.590395 9762 ts_tablet_manager.cc:1888] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.590402 9811 ts_tablet_manager.cc:1888] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.590451 9762 log.cc:1196] T 41b9f1a284c34a9099cf1014db77f35c P d2e209265a41425fba98c9d9b0551bd4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/41b9f1a284c34a9099cf1014db77f35c
I20240501 07:59:59.590461 9811 log.cc:1196] T e952704c6a3d43aba833484bba9fddbf P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/e952704c6a3d43aba833484bba9fddbf
I20240501 07:59:59.591846 8294 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.592254 8607 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.594476 8596 raft_consensus.cc:1050] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4: attempting to promote NON_VOTER 29afadb287c049d9bb777d674b3697ce to VOTER
I20240501 07:59:59.594928 8596 consensus_queue.cc:237] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 07:59:59.596606 6198 raft_consensus.cc:1261] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.596741 8030 raft_consensus.cc:1261] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.597136 9900 consensus_queue.cc:1035] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.597481 9900 consensus_queue.cc:1035] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.597503 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet e952704c6a3d43aba833484bba9fddbf (table rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]) successfully deleted
I20240501 07:59:59.597666 6046 catalog_manager.cc:4830] TS d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501): tablet 41b9f1a284c34a9099cf1014db77f35c (table rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]) successfully deleted
I20240501 07:59:59.601002 6329 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.601261 6593 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.601300 9897 raft_consensus.cc:2932] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.601905 8030 raft_consensus.cc:2932] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.603098 6201 raft_consensus.cc:2932] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.604471 7333 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.604406 6054 catalog_manager.cc:5484] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce reported cstate change: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.604658 9754 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.606690 6463 raft_consensus.cc:1261] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.607148 8596 consensus_queue.cc:1035] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.608641 8031 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.608803 6592 consensus_queue.cc:237] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 07:59:59.610172 6201 raft_consensus.cc:1261] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.610828 7333 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.610960 9900 consensus_queue.cc:1035] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.611760 6329 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.612308 8031 raft_consensus.cc:1261] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.612810 9900 raft_consensus.cc:2932] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.611338 6463 raft_consensus.cc:2932] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.613672 9647 consensus_queue.cc:1035] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.614444 8294 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.615180 8031 raft_consensus.cc:2932] T 7a9dcf4944c34707a8cd0acebc8f9029 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.616376 6051 catalog_manager.cc:5484] T 7a9dcf4944c34707a8cd0acebc8f9029 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:59.616590 6054 catalog_manager.cc:5484] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 reported cstate change: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.617997 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7a9dcf4944c34707a8cd0acebc8f9029 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.618386 6201 raft_consensus.cc:2932] T 7a9dcf4944c34707a8cd0acebc8f9029 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.619020 6593 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.621475 6201 consensus_queue.cc:237] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 07:59:59.621994 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a9dcf4944c34707a8cd0acebc8f9029 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 6)) from {username='slave'} at 127.0.0.1:51996
I20240501 07:59:59.622211 9774 tablet_replica.cc:330] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 07:59:59.622364 9774 raft_consensus.cc:2229] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.622485 9774 raft_consensus.cc:2258] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.622814 8294 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.623097 6329 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.623471 8607 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.623528 9774 ts_tablet_manager.cc:1875] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.625377 7333 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.625792 9774 ts_tablet_manager.cc:1888] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.625859 9774 log.cc:1196] T 7a9dcf4944c34707a8cd0acebc8f9029 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/7a9dcf4944c34707a8cd0acebc8f9029
I20240501 07:59:59.625887 6329 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.626405 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet 7a9dcf4944c34707a8cd0acebc8f9029 (table rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]) successfully deleted
I20240501 07:59:59.626550 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6b552a44c67940819f9ef64c68585eb1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.627216 8607 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.627552 6055 catalog_manager.cc:5484] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:59.628268 8294 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.630592 8607 raft_consensus.cc:1050] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d: attempting to promote NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 to VOTER
I20240501 07:59:59.631001 8607 consensus_queue.cc:237] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 07:59:59.631708 6593 consensus_queue.cc:237] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:59.632622 6592 raft_consensus.cc:1261] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.632939 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b552a44c67940819f9ef64c68585eb1 with delete_type TABLET_DATA_TOMBSTONED (TS d2e209265a41425fba98c9d9b0551bd4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60522
I20240501 07:59:59.633250 7900 raft_consensus.cc:1261] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.633589 9762 tablet_replica.cc:330] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4: stopping tablet replica
I20240501 07:59:59.633724 9762 raft_consensus.cc:2229] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.633831 9762 raft_consensus.cc:2258] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.634418 9762 ts_tablet_manager.cc:1875] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.634575 8607 consensus_queue.cc:1035] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.635308 8607 consensus_queue.cc:1035] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.635520 6201 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.636113 9897 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.636602 6331 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.636951 9762 ts_tablet_manager.cc:1888] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.637063 9762 log.cc:1196] T 6b552a44c67940819f9ef64c68585eb1 P d2e209265a41425fba98c9d9b0551bd4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/6b552a44c67940819f9ef64c68585eb1
I20240501 07:59:59.637712 9897 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.638157 6046 catalog_manager.cc:4830] TS d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501): tablet 6b552a44c67940819f9ef64c68585eb1 (table rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]) successfully deleted
W20240501 07:59:59.638885 6528 consensus_peers.cc:485] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 -> Peer ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937): Couldn't send request to peer ef83010b77b441ffac2e41468a0a1bd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 96386dd2c62e45859e1f3b30510e8969. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:59.638940 8596 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:59.639410 6201 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:59.639609 6329 raft_consensus.cc:1261] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.639954 8607 consensus_queue.cc:1035] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.640779 8607 raft_consensus.cc:2932] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.642346 6329 raft_consensus.cc:2932] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.642381 7900 raft_consensus.cc:2932] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.643074 6051 catalog_manager.cc:5484] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:59.643757 6331 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:59.644613 6057 catalog_manager.cc:5484] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 reported cstate change: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.644994 6592 raft_consensus.cc:2932] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.654469 6331 consensus_queue.cc:237] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 07:59:59.655967 6592 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.655948 6201 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:59.656165 6266 consensus_peers.cc:485] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 -> Peer ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937): Couldn't send request to peer ef83010b77b441ffac2e41468a0a1bd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 71bcfcbbdfc0491ea936168dd699fc92. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:59.656420 8400 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.656628 7817 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.658492 8400 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:59.658876 6592 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:59.659082 6201 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 07:59:59.661536 6055 catalog_manager.cc:5484] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index -1 to 4, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:59.669701 6331 consensus_queue.cc:237] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:59.671300 6463 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.671274 6592 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.671782 7817 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.672058 9763 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.674072 9769 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:59.674475 6592 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:59.675101 6463 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:59.676919 6051 catalog_manager.cc:5484] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 07:59:59.677716 6266 consensus_peers.cc:485] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 -> Peer 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247): Couldn't send request to peer 7a80bdedf813425faf7617b4b07c9f36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a4b3aaeeb72e47e08c938b47310dd402. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:59.682762 9806 raft_consensus.cc:1050] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 to VOTER
I20240501 07:59:59.683596 9806 consensus_queue.cc:237] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 07:59:59.685066 8294 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.685120 6201 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.685532 9246 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.685676 6331 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.685899 9800 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.686107 9246 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.686741 6463 consensus_queue.cc:237] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:59.688072 6198 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:59.688627 6398 consensus_peers.cc:485] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer 29afadb287c049d9bb777d674b3697ce (127.0.99.134:46293): Couldn't send request to peer 29afadb287c049d9bb777d674b3697ce. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98d43e2703b14f9ebf926b5f2e2b2c56. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:59.689278 6329 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.689599 9800 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.690064 9246 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.689968 9805 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.690747 8294 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.690876 6201 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.693480 6057 catalog_manager.cc:5484] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.694644 6329 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.694722 9806 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.696733 6463 consensus_queue.cc:237] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 07:59:59.697216 6329 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.697217 6198 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.699482 8294 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.699995 9246 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.700330 6055 catalog_manager.cc:5484] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.701865 9794 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.702327 8294 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.703416 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4e60dad8ddd2402faae35be18662c8c3 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.704268 6331 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.704677 9805 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.705246 6051 catalog_manager.cc:5484] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 reported cstate change: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.705729 6331 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.708334 6181 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e60dad8ddd2402faae35be18662c8c3 with delete_type TABLET_DATA_TOMBSTONED (TS 953b945e3e7e4c71a61aa99c1208ab1d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45448
I20240501 07:59:59.708585 9768 tablet_replica.cc:330] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d: stopping tablet replica
I20240501 07:59:59.708771 9768 raft_consensus.cc:2229] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.708959 9768 raft_consensus.cc:2258] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.709733 9768 ts_tablet_manager.cc:1875] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.711464 9768 ts_tablet_manager.cc:1888] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.711537 9768 log.cc:1196] T 4e60dad8ddd2402faae35be18662c8c3 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/4e60dad8ddd2402faae35be18662c8c3
I20240501 07:59:59.712049 6046 catalog_manager.cc:4830] TS 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189): tablet 4e60dad8ddd2402faae35be18662c8c3 (table rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]) successfully deleted
I20240501 07:59:59.715051 6592 consensus_queue.cc:237] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:59.717389 6198 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.717633 6331 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.717820 9897 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.718489 7817 raft_consensus.cc:1050] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 to VOTER
I20240501 07:59:59.718704 9897 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.718879 7817 consensus_queue.cc:237] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 07:59:59.720279 8294 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.720723 8400 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.721195 6592 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.721921 8400 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.722110 6463 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.722640 9897 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:59.723114 8400 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 07:59:59.723575 6527 consensus_peers.cc:485] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 -> Peer 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136:40449): Couldn't send request to peer 06c63579c7234194b5f6ef457dd5e3f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2bbceded7f37434ca25b1068bdfc8ca1. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:59.724781 6331 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:59.725436 9763 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.725891 6463 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.725924 6057 catalog_manager.cc:5484] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:59.728485 6051 catalog_manager.cc:5484] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.729593 8294 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.730909 6198 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:59.734369 6592 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.739513 6331 consensus_queue.cc:237] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 07:59:59.741148 8294 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.741148 6592 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.741705 9763 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.741920 9964 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.743772 8400 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.744254 8294 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.744387 6592 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 07:59:59.745663 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 88fa1c4aaf754e32bef123157b3bc62c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.746701 6057 catalog_manager.cc:5484] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.749617 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet 88fa1c4aaf754e32bef123157b3bc62c with delete_type TABLET_DATA_TOMBSTONED (TS 5db2970f4bbd4a3c92d7d913f99716b1 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:51996
I20240501 07:59:59.749878 9774 tablet_replica.cc:330] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 07:59:59.750032 9774 raft_consensus.cc:2229] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.750207 9774 raft_consensus.cc:2258] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.750515 6463 consensus_queue.cc:237] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 07:59:59.752359 9774 ts_tablet_manager.cc:1875] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.752754 9805 raft_consensus.cc:1050] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER 29afadb287c049d9bb777d674b3697ce to VOTER
I20240501 07:59:59.753132 9805 consensus_queue.cc:237] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 07:59:59.754562 9774 ts_tablet_manager.cc:1888] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.754637 9774 log.cc:1196] T 88fa1c4aaf754e32bef123157b3bc62c P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/88fa1c4aaf754e32bef123157b3bc62c
I20240501 07:59:59.754657 8031 raft_consensus.cc:1261] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.754657 6329 raft_consensus.cc:1261] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.755071 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet 88fa1c4aaf754e32bef123157b3bc62c (table rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]) successfully deleted
I20240501 07:59:59.755111 9794 consensus_queue.cc:1035] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.755383 6331 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.755421 8617 consensus_queue.cc:1035] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.755656 6198 raft_consensus.cc:1261] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.755746 6592 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:59.756304 6397 consensus_peers.cc:485] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247): Couldn't send request to peer 7a80bdedf813425faf7617b4b07c9f36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33863ce53ec14aab9040f75e0360e0af. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:59.756388 9800 consensus_queue.cc:1035] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.756691 9800 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.757005 9800 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.757881 9794 raft_consensus.cc:2932] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.760002 8031 raft_consensus.cc:2932] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.760380 6331 raft_consensus.cc:2932] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.760499 8617 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:59.761070 6198 raft_consensus.cc:2932] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.762936 6055 catalog_manager.cc:5484] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.763499 6592 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:59.765004 6331 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:59.767748 6051 catalog_manager.cc:5484] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 07:59:59.772037 6463 consensus_queue.cc:237] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 07:59:59.773063 6198 raft_consensus.cc:1261] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.773149 8031 raft_consensus.cc:1261] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.773443 9805 consensus_queue.cc:1035] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.773615 9794 consensus_queue.cc:1035] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.775777 9806 raft_consensus.cc:2932] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.776599 6198 raft_consensus.cc:2932] T 47d3021e3ac74c6ba482b8b54066c36e P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.776937 8031 raft_consensus.cc:2932] T 47d3021e3ac74c6ba482b8b54066c36e P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.777549 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 47d3021e3ac74c6ba482b8b54066c36e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.778893 6055 catalog_manager.cc:5484] T 47d3021e3ac74c6ba482b8b54066c36e P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.781800 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet 47d3021e3ac74c6ba482b8b54066c36e with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60224
I20240501 07:59:59.782131 9811 tablet_replica.cc:330] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 07:59:59.782361 9811 raft_consensus.cc:2229] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.782560 9811 raft_consensus.cc:2258] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.783658 9811 ts_tablet_manager.cc:1875] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.786026 9811 ts_tablet_manager.cc:1888] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.786108 9811 log.cc:1196] T 47d3021e3ac74c6ba482b8b54066c36e P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/47d3021e3ac74c6ba482b8b54066c36e
I20240501 07:59:59.786623 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet 47d3021e3ac74c6ba482b8b54066c36e (table rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]) successfully deleted
I20240501 07:59:59.848495 7333 raft_consensus.cc:1050] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d: attempting to promote NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 to VOTER
I20240501 07:59:59.848944 7333 consensus_queue.cc:237] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 07:59:59.850574 6592 raft_consensus.cc:1261] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.850668 6463 raft_consensus.cc:1261] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.851029 9752 consensus_queue.cc:1035] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.851109 7900 raft_consensus.cc:1261] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.851389 9752 consensus_queue.cc:1035] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.851547 8607 consensus_queue.cc:1035] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.855146 9754 raft_consensus.cc:2932] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.855635 6592 raft_consensus.cc:2932] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.855669 6463 raft_consensus.cc:2932] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.858083 7900 raft_consensus.cc:2932] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.858583 6057 catalog_manager.cc:5484] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.862668 6198 consensus_queue.cc:237] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 07:59:59.863888 6463 raft_consensus.cc:1261] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.864178 7900 raft_consensus.cc:1261] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 07:59:59.864326 8607 consensus_queue.cc:1035] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.864696 8607 consensus_queue.cc:1035] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 07:59:59.866091 9754 raft_consensus.cc:2932] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.867516 7900 raft_consensus.cc:2932] T 5d0ae7c274af4ce1992e9507cbabc551 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.867832 6463 raft_consensus.cc:2932] T 5d0ae7c274af4ce1992e9507cbabc551 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 07:59:59.868211 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5d0ae7c274af4ce1992e9507cbabc551 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 07:59:59.869520 6055 catalog_manager.cc:5484] T 5d0ae7c274af4ce1992e9507cbabc551 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 07:59:59.872960 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d0ae7c274af4ce1992e9507cbabc551 with delete_type TABLET_DATA_TOMBSTONED (TS d2e209265a41425fba98c9d9b0551bd4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60522
I20240501 07:59:59.873225 9762 tablet_replica.cc:330] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4: stopping tablet replica
I20240501 07:59:59.873397 9762 raft_consensus.cc:2229] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 07:59:59.873574 9762 raft_consensus.cc:2258] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 07:59:59.874219 9762 ts_tablet_manager.cc:1875] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 07:59:59.876190 9762 ts_tablet_manager.cc:1888] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 07:59:59.876259 9762 log.cc:1196] T 5d0ae7c274af4ce1992e9507cbabc551 P d2e209265a41425fba98c9d9b0551bd4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/5d0ae7c274af4ce1992e9507cbabc551
I20240501 07:59:59.876744 6046 catalog_manager.cc:4830] TS d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501): tablet 5d0ae7c274af4ce1992e9507cbabc551 (table rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]) successfully deleted
I20240501 07:59:59.879993 6198 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5dae6c0b9c1d4009a93ed157e1f763c8"
dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52614
I20240501 07:59:59.880164 6198 raft_consensus.cc:595] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Received request to transfer leadership
I20240501 07:59:59.913640 6593 consensus_queue.cc:237] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 07:59:59.915123 6198 raft_consensus.cc:1261] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.915120 6463 raft_consensus.cc:1261] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:59.915324 6527 consensus_peers.cc:485] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 -> Peer 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136:40449): Couldn't send request to peer 06c63579c7234194b5f6ef457dd5e3f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dbc4b418e4c5493fa8a7734740225200. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:59.915709 8596 consensus_queue.cc:1035] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.915897 9900 consensus_queue.cc:1035] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.917807 8596 raft_consensus.cc:2932] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:59.918292 6463 raft_consensus.cc:2932] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:59.920742 6198 raft_consensus.cc:2932] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 07:59:59.920964 6057 catalog_manager.cc:5484] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index -1 to 4, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:59.929286 6593 consensus_queue.cc:237] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:59.930851 6331 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:59.930950 6529 consensus_peers.cc:485] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 -> Peer 29afadb287c049d9bb777d674b3697ce (127.0.99.134:46293): Couldn't send request to peer 29afadb287c049d9bb777d674b3697ce. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e052973ae7744e8887d3df0d420f411. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:59.931026 6198 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.931290 9900 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.931836 9900 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.933126 9897 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.933761 6331 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.934442 6198 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.936910 6057 catalog_manager.cc:5484] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:59.944813 6198 consensus_queue.cc:237] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:59.946342 6331 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.946424 6463 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:59.946455 6136 consensus_peers.cc:485] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d -> Peer 29afadb287c049d9bb777d674b3697ce (127.0.99.134:46293): Couldn't send request to peer 29afadb287c049d9bb777d674b3697ce. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a97136a7017541eb8e4afb281c97e3cb. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:59.946777 8607 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.946987 9754 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.948920 8607 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.949467 6463 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.949548 6331 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.952608 6055 catalog_manager.cc:5484] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.960078 6593 consensus_queue.cc:237] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 07:59:59.961701 6198 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 07:59:59.961807 6463 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 07:59:59.961814 6529 consensus_peers.cc:485] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 -> Peer 29afadb287c049d9bb777d674b3697ce (127.0.99.134:46293): Couldn't send request to peer 29afadb287c049d9bb777d674b3697ce. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6bf00ed2af0e42ca94c99a701fbe7dd1. This is attempt 1: this message will repeat every 5th retry.
I20240501 07:59:59.962124 9897 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.962446 9897 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 07:59:59.963872 9897 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.964514 6463 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.966496 6051 catalog_manager.cc:5484] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 07:59:59.965893 6198 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 07:59:59.982941 9896 raft_consensus.cc:1050] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d: attempting to promote NON_VOTER 29afadb287c049d9bb777d674b3697ce to VOTER
I20240501 07:59:59.983431 9896 consensus_queue.cc:237] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 07:59:59.985164 6463 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.985181 6331 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.985224 8031 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 07:59:59.985595 7333 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.985785 9754 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.986094 7333 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 07:59:59.989076 7333 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.989846 6463 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.989979 6331 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.991237 8031 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.993815 6055 catalog_manager.cc:5484] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce reported cstate change: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 07:59:59.997920 6198 consensus_queue.cc:237] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:00.000001 6463 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:00.000178 8031 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:00.000705 9752 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:00.000984 9754 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:00.003515 9754 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:00.003841 6463 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:00.003960 8031 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:00.006882 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d929e6e09be943b68061ad4b1bb8e9f1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:00.007117 6051 catalog_manager.cc:5484] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:00.010288 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet d929e6e09be943b68061ad4b1bb8e9f1 with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60224
I20240501 08:00:00.010532 9811 tablet_replica.cc:330] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 08:00:00.010679 9811 raft_consensus.cc:2229] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:00.010855 9811 raft_consensus.cc:2258] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:00.011515 9811 ts_tablet_manager.cc:1875] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:00.013861 9811 ts_tablet_manager.cc:1888] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:00.013955 9811 log.cc:1196] T d929e6e09be943b68061ad4b1bb8e9f1 P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/d929e6e09be943b68061ad4b1bb8e9f1
I20240501 08:00:00.014443 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet d929e6e09be943b68061ad4b1bb8e9f1 (table rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]) successfully deleted
I20240501 08:00:00.033948 9763 raft_consensus.cc:1050] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 to VOTER
I20240501 08:00:00.034406 9763 consensus_queue.cc:237] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:00.035713 8162 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:00.035779 6593 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:00.035877 6463 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:00.036178 9769 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:00.036375 9964 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:00.036665 7817 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:00.039276 9964 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.039669 8162 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.039824 6463 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.042531 6051 catalog_manager.cc:5484] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.044621 6593 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.045821 6331 consensus_queue.cc:237] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:00.047726 8162 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:00.048166 9769 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:00.048840 6593 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:00.049239 9769 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:00.049878 9763 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.050338 8162 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.051425 6593 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.052986 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f4c62996a5a64d978850f2f7a31009f4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:00.053161 6057 catalog_manager.cc:5484] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:00.055586 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet f4c62996a5a64d978850f2f7a31009f4 with delete_type TABLET_DATA_TOMBSTONED (TS 5db2970f4bbd4a3c92d7d913f99716b1 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:51996
I20240501 08:00:00.055814 9774 tablet_replica.cc:330] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 08:00:00.055982 9774 raft_consensus.cc:2229] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:00.056138 9774 raft_consensus.cc:2258] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:00.056886 9774 ts_tablet_manager.cc:1875] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:00.058710 9774 ts_tablet_manager.cc:1888] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:00.058763 9774 log.cc:1196] T f4c62996a5a64d978850f2f7a31009f4 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/f4c62996a5a64d978850f2f7a31009f4
I20240501 08:00:00.059240 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet f4c62996a5a64d978850f2f7a31009f4 (table rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]) successfully deleted
I20240501 08:00:00.101819 10201 ts_tablet_manager.cc:902] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 08:00:00.101871 6198 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5dae6c0b9c1d4009a93ed157e1f763c8"
dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52660
I20240501 08:00:00.102000 6198 raft_consensus.cc:595] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:00.102355 10201 tablet_copy_client.cc:308] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 08:00:00.102835 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet 71bcfcbbdfc0491ea936168dd699fc92 from peer ef83010b77b441ffac2e41468a0a1bd1 ({username='slave'} at 127.0.99.135:48397)
I20240501 08:00:00.102939 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet 71bcfcbbdfc0491ea936168dd699fc92 from peer ef83010b77b441ffac2e41468a0a1bd1 at {username='slave'} at 127.0.99.135:48397: session id = ef83010b77b441ffac2e41468a0a1bd1-71bcfcbbdfc0491ea936168dd699fc92
I20240501 08:00:00.103789 6352 tablet_copy_source_session.cc:205] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:00.104646 10201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71bcfcbbdfc0491ea936168dd699fc92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:00.106892 10201 tablet_copy_client.cc:783] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:00.107044 10201 tablet_copy_client.cc:647] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:00.108327 10202 tablet_copy_client.cc:1002] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:00.115187 9794 raft_consensus.cc:1050] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 to VOTER
I20240501 08:00:00.115622 9794 consensus_queue.cc:237] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:00.117022 6198 raft_consensus.cc:1261] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:00.117025 7900 raft_consensus.cc:1261] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:00.117024 6593 raft_consensus.cc:1261] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:00.117362 9794 consensus_queue.cc:1035] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:00.117556 9805 consensus_queue.cc:1035] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:00.117722 9800 consensus_queue.cc:1035] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:00.120597 9794 raft_consensus.cc:2932] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:00.120798 6463 consensus_queue.cc:237] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:00.121202 7900 raft_consensus.cc:2932] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:00.123200 6201 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:00.123381 6593 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:00.123893 9806 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:00.124119 9800 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:00.124361 6053 catalog_manager.cc:5484] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:00.124972 6593 raft_consensus.cc:2932] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:00.125499 6198 raft_consensus.cc:2932] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:00.127115 9805 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 08:00:00.127580 6592 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
W20240501 08:00:00.128392 6397 consensus_peers.cc:485] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247): Couldn't send request to peer 7a80bdedf813425faf7617b4b07c9f36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5c411a11ec484cd5bd063e5fc3868e29. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:00.130035 8607 raft_consensus.cc:979] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d: : Instructing follower 7a80bdedf813425faf7617b4b07c9f36 to start an election
I20240501 08:00:00.130143 7333 raft_consensus.cc:1067] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Signalling peer 7a80bdedf813425faf7617b4b07c9f36 to start an election
I20240501 08:00:00.131659 6463 consensus_queue.cc:237] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:00.132378 6053 catalog_manager.cc:5484] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 08:00:00.132764 6593 raft_consensus.cc:1261] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:00.132709 7900 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5dae6c0b9c1d4009a93ed157e1f763c8"
dest_uuid: "7a80bdedf813425faf7617b4b07c9f36"
from {username='slave'} at 127.0.99.129:60891
I20240501 08:00:00.132784 7899 raft_consensus.cc:1261] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:00.133265 9800 consensus_queue.cc:1035] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:00.133539 9805 consensus_queue.cc:1035] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:00.133986 7900 raft_consensus.cc:482] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:00.134227 7900 raft_consensus.cc:3037] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:00.135742 9246 raft_consensus.cc:2932] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:00.136246 6593 raft_consensus.cc:2932] T 947b5d16224a42dcad709e7ddefc0b33 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:00.137257 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 947b5d16224a42dcad709e7ddefc0b33 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:00.138406 7900 raft_consensus.cc:504] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:00.138784 6051 catalog_manager.cc:5484] T 947b5d16224a42dcad709e7ddefc0b33 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:00.139881 7900 leader_election.cc:288] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [CANDIDATE]: Term 2 election: Requested vote from peers 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 08:00:00.141304 6198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dae6c0b9c1d4009a93ed157e1f763c8" candidate_uuid: "7a80bdedf813425faf7617b4b07c9f36" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 08:00:00.141429 6198 raft_consensus.cc:3032] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:00:00.141469 6198 raft_consensus.cc:729] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Running, Role: LEADER
I20240501 08:00:00.141619 6198 consensus_queue.cc:260] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:00.141678 6181 tablet_service.cc:1514] Processing DeleteTablet for tablet 947b5d16224a42dcad709e7ddefc0b33 with delete_type TABLET_DATA_TOMBSTONED (TS 953b945e3e7e4c71a61aa99c1208ab1d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45448
I20240501 08:00:00.141831 6198 raft_consensus.cc:3037] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:00.141955 9768 tablet_replica.cc:330] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d: stopping tablet replica
I20240501 08:00:00.142122 9768 raft_consensus.cc:2229] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:00.142259 9768 raft_consensus.cc:2258] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:00.142988 6198 raft_consensus.cc:2454] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a80bdedf813425faf7617b4b07c9f36 in term 2.
I20240501 08:00:00.143244 9768 ts_tablet_manager.cc:1875] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:00.144061 10230 ts_tablet_manager.cc:902] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 08:00:00.144604 10230 tablet_copy_client.cc:308] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 08:00:00.145200 6198 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 08:00:00.146085 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dae6c0b9c1d4009a93ed157e1f763c8" candidate_uuid: "7a80bdedf813425faf7617b4b07c9f36" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 08:00:00.146204 6331 raft_consensus.cc:3037] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:00.146109 7899 raft_consensus.cc:2932] T 947b5d16224a42dcad709e7ddefc0b33 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:00.147274 6331 raft_consensus.cc:2454] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a80bdedf813425faf7617b4b07c9f36 in term 2.
I20240501 08:00:00.147485 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet a4b3aaeeb72e47e08c938b47310dd402 from peer 7a80bdedf813425faf7617b4b07c9f36 ({username='slave'} at 127.0.99.133:33299)
I20240501 08:00:00.147580 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet a4b3aaeeb72e47e08c938b47310dd402 from peer 7a80bdedf813425faf7617b4b07c9f36 at {username='slave'} at 127.0.99.133:33299: session id = 7a80bdedf813425faf7617b4b07c9f36-a4b3aaeeb72e47e08c938b47310dd402
I20240501 08:00:00.148104 7836 leader_election.cc:302] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 7a80bdedf813425faf7617b4b07c9f36, 953b945e3e7e4c71a61aa99c1208ab1d; no voters:
I20240501 08:00:00.148327 9569 raft_consensus.cc:2781] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:00:00.148527 9569 raft_consensus.cc:686] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 LEADER]: Becoming Leader. State: Replica: 7a80bdedf813425faf7617b4b07c9f36, State: Running, Role: LEADER
I20240501 08:00:00.148861 9569 consensus_queue.cc:237] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:00.148962 6352 tablet_copy_source_session.cc:205] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:00.149912 10230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b3aaeeb72e47e08c938b47310dd402. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:00.149930 9768 ts_tablet_manager.cc:1888] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:00.150004 9768 log.cc:1196] T 947b5d16224a42dcad709e7ddefc0b33 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/947b5d16224a42dcad709e7ddefc0b33
I20240501 08:00:00.150987 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dae6c0b9c1d4009a93ed157e1f763c8" candidate_uuid: "7a80bdedf813425faf7617b4b07c9f36" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 08:00:00.151120 6593 raft_consensus.cc:3037] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:00.152117 6593 raft_consensus.cc:2454] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a80bdedf813425faf7617b4b07c9f36 in term 2.
I20240501 08:00:00.152468 6046 catalog_manager.cc:4830] TS 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189): tablet 947b5d16224a42dcad709e7ddefc0b33 (table rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]) successfully deleted
I20240501 08:00:00.152842 10230 tablet_copy_client.cc:783] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:00.153142 10230 tablet_copy_client.cc:647] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:00.154127 10236 ts_tablet_manager.cc:902] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8: Initiating tablet copy from peer d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 08:00:00.154222 10231 tablet_copy_client.cc:1002] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:00.154371 6051 catalog_manager.cc:5484] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 reported cstate change: term changed from 1 to 2, leader changed from 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) to 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133). New cstate: current_term: 2 leader_uuid: "7a80bdedf813425faf7617b4b07c9f36" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:00.154712 10236 tablet_copy_client.cc:308] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:42501
I20240501 08:00:00.159173 6614 tablet_copy_service.cc:140] P d2e209265a41425fba98c9d9b0551bd4: Received BeginTabletCopySession request for tablet 2bbceded7f37434ca25b1068bdfc8ca1 from peer 06c63579c7234194b5f6ef457dd5e3f8 ({username='slave'} at 127.0.99.136:45667)
I20240501 08:00:00.159312 6614 tablet_copy_service.cc:161] P d2e209265a41425fba98c9d9b0551bd4: Beginning new tablet copy session on tablet 2bbceded7f37434ca25b1068bdfc8ca1 from peer 06c63579c7234194b5f6ef457dd5e3f8 at {username='slave'} at 127.0.99.136:45667: session id = 06c63579c7234194b5f6ef457dd5e3f8-2bbceded7f37434ca25b1068bdfc8ca1
I20240501 08:00:00.160106 6614 tablet_copy_source_session.cc:205] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:00.160979 10236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bbceded7f37434ca25b1068bdfc8ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:00.163028 7817 raft_consensus.cc:1050] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 to VOTER
I20240501 08:00:00.163462 7817 consensus_queue.cc:237] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:00.163645 10236 tablet_copy_client.cc:783] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:00.163801 10236 tablet_copy_client.cc:647] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:00.164804 10240 tablet_copy_client.cc:1002] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:00.165045 8162 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:00.165201 6593 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:00.165433 9769 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:00.165601 6463 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:00.165607 9763 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:00.165962 9769 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:00.167618 9769 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.168320 6593 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.168320 6463 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.169508 8162 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.170941 6053 catalog_manager.cc:5484] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.177395 6331 consensus_queue.cc:237] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:00.178761 8162 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:00.178831 6593 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:00.179141 7817 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:00.179347 9763 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:00.181105 9763 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.181537 6593 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.181710 8162 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:00.182920 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet de48f8c2870f4517b5349c30a51fce56 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:00.183956 6053 catalog_manager.cc:5484] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:00.186759 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet de48f8c2870f4517b5349c30a51fce56 with delete_type TABLET_DATA_TOMBSTONED (TS 5db2970f4bbd4a3c92d7d913f99716b1 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:51996
I20240501 08:00:00.187038 9774 tablet_replica.cc:330] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 08:00:00.187186 9774 raft_consensus.cc:2229] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:00.187357 9774 raft_consensus.cc:2258] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:00.188201 9774 ts_tablet_manager.cc:1875] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:00.190531 9774 ts_tablet_manager.cc:1888] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:00.190598 9774 log.cc:1196] T de48f8c2870f4517b5349c30a51fce56 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/de48f8c2870f4517b5349c30a51fce56
I20240501 08:00:00.191140 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet de48f8c2870f4517b5349c30a51fce56 (table rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]) successfully deleted
I20240501 08:00:00.209879 10276 ts_tablet_manager.cc:902] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 08:00:00.210402 10276 tablet_copy_client.cc:308] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
I20240501 08:00:00.210978 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet 33863ce53ec14aab9040f75e0360e0af from peer 7a80bdedf813425faf7617b4b07c9f36 ({username='slave'} at 127.0.99.133:60211)
I20240501 08:00:00.211086 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet 33863ce53ec14aab9040f75e0360e0af from peer 7a80bdedf813425faf7617b4b07c9f36 at {username='slave'} at 127.0.99.133:60211: session id = 7a80bdedf813425faf7617b4b07c9f36-33863ce53ec14aab9040f75e0360e0af
I20240501 08:00:00.212018 6483 tablet_copy_source_session.cc:205] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:00.213110 10276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33863ce53ec14aab9040f75e0360e0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:00.216221 10276 tablet_copy_client.cc:783] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:00.216424 10276 tablet_copy_client.cc:647] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:00.217550 10284 tablet_copy_client.cc:1002] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:00.243039 10311 ts_tablet_manager.cc:902] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: Initiating tablet copy from peer d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 08:00:00.243577 10311 tablet_copy_client.cc:308] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:42501
I20240501 08:00:00.248134 6614 tablet_copy_service.cc:140] P d2e209265a41425fba98c9d9b0551bd4: Received BeginTabletCopySession request for tablet 96386dd2c62e45859e1f3b30510e8969 from peer ef83010b77b441ffac2e41468a0a1bd1 ({username='slave'} at 127.0.99.135:54555)
I20240501 08:00:00.248250 6614 tablet_copy_service.cc:161] P d2e209265a41425fba98c9d9b0551bd4: Beginning new tablet copy session on tablet 96386dd2c62e45859e1f3b30510e8969 from peer ef83010b77b441ffac2e41468a0a1bd1 at {username='slave'} at 127.0.99.135:54555: session id = ef83010b77b441ffac2e41468a0a1bd1-96386dd2c62e45859e1f3b30510e8969
I20240501 08:00:00.249024 6614 tablet_copy_source_session.cc:205] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:00.249864 10311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96386dd2c62e45859e1f3b30510e8969. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:00.251683 10311 tablet_copy_client.cc:783] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:00.251827 10311 tablet_copy_client.cc:647] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:00.252948 10313 tablet_copy_client.cc:1002] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:00.294888 10353 ts_tablet_manager.cc:902] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 08:00:00.295521 10353 tablet_copy_client.cc:308] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
I20240501 08:00:00.296034 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet 98d43e2703b14f9ebf926b5f2e2b2c56 from peer 29afadb287c049d9bb777d674b3697ce ({username='slave'} at 127.0.99.134:36071)
I20240501 08:00:00.296144 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet 98d43e2703b14f9ebf926b5f2e2b2c56 from peer 29afadb287c049d9bb777d674b3697ce at {username='slave'} at 127.0.99.134:36071: session id = 29afadb287c049d9bb777d674b3697ce-98d43e2703b14f9ebf926b5f2e2b2c56
I20240501 08:00:00.296917 6483 tablet_copy_source_session.cc:205] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:00.297713 10353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98d43e2703b14f9ebf926b5f2e2b2c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:00.300103 10353 tablet_copy_client.cc:783] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:00.300307 10353 tablet_copy_client.cc:647] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:00.301525 10354 tablet_copy_client.cc:1002] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:00.414739 10388 ts_tablet_manager.cc:902] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce: Initiating tablet copy from peer d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 08:00:00.415395 10388 tablet_copy_client.cc:308] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:42501
I20240501 08:00:00.415872 6614 tablet_copy_service.cc:140] P d2e209265a41425fba98c9d9b0551bd4: Received BeginTabletCopySession request for tablet 4e052973ae7744e8887d3df0d420f411 from peer 29afadb287c049d9bb777d674b3697ce ({username='slave'} at 127.0.99.134:43243)
I20240501 08:00:00.416000 6614 tablet_copy_service.cc:161] P d2e209265a41425fba98c9d9b0551bd4: Beginning new tablet copy session on tablet 4e052973ae7744e8887d3df0d420f411 from peer 29afadb287c049d9bb777d674b3697ce at {username='slave'} at 127.0.99.134:43243: session id = 29afadb287c049d9bb777d674b3697ce-4e052973ae7744e8887d3df0d420f411
I20240501 08:00:00.416759 6614 tablet_copy_source_session.cc:205] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:00.417595 10388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e052973ae7744e8887d3df0d420f411. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:00.419356 10388 tablet_copy_client.cc:783] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:00.419494 10388 tablet_copy_client.cc:647] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:00.420490 10392 tablet_copy_client.cc:1002] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:00.438963 10393 ts_tablet_manager.cc:902] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8: Initiating tablet copy from peer d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 08:00:00.439479 10393 tablet_copy_client.cc:308] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:42501
I20240501 08:00:00.440110 6614 tablet_copy_service.cc:140] P d2e209265a41425fba98c9d9b0551bd4: Received BeginTabletCopySession request for tablet dbc4b418e4c5493fa8a7734740225200 from peer 06c63579c7234194b5f6ef457dd5e3f8 ({username='slave'} at 127.0.99.136:45667)
I20240501 08:00:00.440232 6614 tablet_copy_service.cc:161] P d2e209265a41425fba98c9d9b0551bd4: Beginning new tablet copy session on tablet dbc4b418e4c5493fa8a7734740225200 from peer 06c63579c7234194b5f6ef457dd5e3f8 at {username='slave'} at 127.0.99.136:45667: session id = 06c63579c7234194b5f6ef457dd5e3f8-dbc4b418e4c5493fa8a7734740225200
I20240501 08:00:00.441171 6614 tablet_copy_source_session.cc:205] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:00.442102 10393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbc4b418e4c5493fa8a7734740225200. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:00.443900 10393 tablet_copy_client.cc:783] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:00.444183 10393 tablet_copy_client.cc:647] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:00.445405 10397 tablet_copy_client.cc:1002] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:00.503165 10398 ts_tablet_manager.cc:902] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce: Initiating tablet copy from peer 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 08:00:00.503648 10398 tablet_copy_client.cc:308] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:44189
I20240501 08:00:00.504138 6220 tablet_copy_service.cc:140] P 953b945e3e7e4c71a61aa99c1208ab1d: Received BeginTabletCopySession request for tablet a97136a7017541eb8e4afb281c97e3cb from peer 29afadb287c049d9bb777d674b3697ce ({username='slave'} at 127.0.99.134:42931)
I20240501 08:00:00.504243 6220 tablet_copy_service.cc:161] P 953b945e3e7e4c71a61aa99c1208ab1d: Beginning new tablet copy session on tablet a97136a7017541eb8e4afb281c97e3cb from peer 29afadb287c049d9bb777d674b3697ce at {username='slave'} at 127.0.99.134:42931: session id = 29afadb287c049d9bb777d674b3697ce-a97136a7017541eb8e4afb281c97e3cb
I20240501 08:00:00.504933 6220 tablet_copy_source_session.cc:205] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:00.505769 10398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97136a7017541eb8e4afb281c97e3cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:00.507552 10398 tablet_copy_client.cc:783] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:00.507735 10398 tablet_copy_client.cc:647] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:00.508630 10402 tablet_copy_client.cc:1002] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:00.521373 10403 ts_tablet_manager.cc:902] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 08:00:00.521914 10403 tablet_copy_client.cc:308] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
I20240501 08:00:00.522430 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet 5c411a11ec484cd5bd063e5fc3868e29 from peer 7a80bdedf813425faf7617b4b07c9f36 ({username='slave'} at 127.0.99.133:60211)
I20240501 08:00:00.522543 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet 5c411a11ec484cd5bd063e5fc3868e29 from peer 7a80bdedf813425faf7617b4b07c9f36 at {username='slave'} at 127.0.99.133:60211: session id = 7a80bdedf813425faf7617b4b07c9f36-5c411a11ec484cd5bd063e5fc3868e29
I20240501 08:00:00.523294 6483 tablet_copy_source_session.cc:205] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:00.524149 10403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c411a11ec484cd5bd063e5fc3868e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:00.525843 10403 tablet_copy_client.cc:783] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:00.526072 10403 tablet_copy_client.cc:647] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:00.527340 10407 tablet_copy_client.cc:1002] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:00.582453 10443 ts_tablet_manager.cc:902] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce: Initiating tablet copy from peer d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 08:00:00.583182 10443 tablet_copy_client.cc:308] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:42501
I20240501 08:00:00.584055 6614 tablet_copy_service.cc:140] P d2e209265a41425fba98c9d9b0551bd4: Received BeginTabletCopySession request for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 from peer 29afadb287c049d9bb777d674b3697ce ({username='slave'} at 127.0.99.134:43243)
I20240501 08:00:00.584168 6614 tablet_copy_service.cc:161] P d2e209265a41425fba98c9d9b0551bd4: Beginning new tablet copy session on tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 from peer 29afadb287c049d9bb777d674b3697ce at {username='slave'} at 127.0.99.134:43243: session id = 29afadb287c049d9bb777d674b3697ce-6bf00ed2af0e42ca94c99a701fbe7dd1
I20240501 08:00:00.585031 6614 tablet_copy_source_session.cc:205] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:00.586860 10443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:00.589628 10443 tablet_copy_client.cc:783] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:00.589776 10443 tablet_copy_client.cc:647] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:00.591122 10447 tablet_copy_client.cc:1002] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:00.650069 6329 raft_consensus.cc:1261] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer 7a80bdedf813425faf7617b4b07c9f36: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 08:00:00.650607 10235 consensus_queue.cc:1035] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:00.734699 6198 raft_consensus.cc:1261] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Refusing update from remote peer 7a80bdedf813425faf7617b4b07c9f36: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 08:00:00.735244 10235 consensus_queue.cc:1035] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:00.738734 6593 raft_consensus.cc:1261] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Refusing update from remote peer 7a80bdedf813425faf7617b4b07c9f36: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 08:00:00.740067 10235 consensus_queue.cc:1035] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:01.199352 7899 consensus_queue.cc:237] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:01.200671 6592 raft_consensus.cc:1261] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Refusing update from remote peer 7a80bdedf813425faf7617b4b07c9f36: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 08:00:01.200767 6329 raft_consensus.cc:1261] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer 7a80bdedf813425faf7617b4b07c9f36: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 08:00:01.201272 10477 consensus_queue.cc:1035] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:01.201510 10235 consensus_queue.cc:1035] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:01.203455 10481 raft_consensus.cc:2932] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:01.203877 6592 raft_consensus.cc:2932] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:01.203979 6329 raft_consensus.cc:2932] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:01.205483 6044 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5dae6c0b9c1d4009a93ed157e1f763c8 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:01.206770 6051 catalog_manager.cc:5484] T 5dae6c0b9c1d4009a93ed157e1f763c8 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 5 to 7, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "7a80bdedf813425faf7617b4b07c9f36" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:01.212249 6181 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dae6c0b9c1d4009a93ed157e1f763c8 with delete_type TABLET_DATA_TOMBSTONED (TS 953b945e3e7e4c71a61aa99c1208ab1d not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:45448
I20240501 08:00:01.212642 10613 tablet_replica.cc:330] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d: stopping tablet replica
I20240501 08:00:01.212791 10613 raft_consensus.cc:2229] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:01.213033 10613 raft_consensus.cc:2258] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:01.214073 10613 ts_tablet_manager.cc:1875] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:01.215914 10613 ts_tablet_manager.cc:1888] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240501 08:00:01.216044 10613 log.cc:1196] T 5dae6c0b9c1d4009a93ed157e1f763c8 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/5dae6c0b9c1d4009a93ed157e1f763c8
I20240501 08:00:01.216745 6046 catalog_manager.cc:4830] TS 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189): tablet 5dae6c0b9c1d4009a93ed157e1f763c8 (table rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]) successfully deleted
I20240501 08:00:01.608697 10201 tablet_copy_client.cc:523] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:01.610100 10201 tablet_bootstrap.cc:492] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap starting.
I20240501 08:00:01.629580 10201 tablet_bootstrap.cc:492] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:01.629940 10201 tablet_bootstrap.cc:492] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap complete.
I20240501 08:00:01.630148 10201 ts_tablet_manager.cc:1367] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1: Time spent bootstrapping tablet: real 0.020s user 0.014s sys 0.005s
I20240501 08:00:01.630662 10201 raft_consensus.cc:348] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:01.630775 10201 raft_consensus.cc:729] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Initialized, Role: LEARNER
I20240501 08:00:01.630852 10201 consensus_queue.cc:260] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:01.631141 10201 ts_tablet_manager.cc:1398] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 08:00:01.631906 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session ef83010b77b441ffac2e41468a0a1bd1-71bcfcbbdfc0491ea936168dd699fc92 received from {username='slave'} at 127.0.99.135:48397
I20240501 08:00:01.632042 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session ef83010b77b441ffac2e41468a0a1bd1-71bcfcbbdfc0491ea936168dd699fc92 on tablet 71bcfcbbdfc0491ea936168dd699fc92 with peer ef83010b77b441ffac2e41468a0a1bd1
I20240501 08:00:01.654592 10230 tablet_copy_client.cc:523] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:01.656244 10230 tablet_bootstrap.cc:492] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap starting.
I20240501 08:00:01.665169 10236 tablet_copy_client.cc:523] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:01.666704 10236 tablet_bootstrap.cc:492] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap starting.
I20240501 08:00:01.677814 10230 tablet_bootstrap.cc:492] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:01.678169 10230 tablet_bootstrap.cc:492] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap complete.
I20240501 08:00:01.678370 10230 ts_tablet_manager.cc:1367] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36: Time spent bootstrapping tablet: real 0.022s user 0.020s sys 0.000s
I20240501 08:00:01.678915 10230 raft_consensus.cc:348] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:01.679069 10230 raft_consensus.cc:729] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a80bdedf813425faf7617b4b07c9f36, State: Initialized, Role: LEARNER
I20240501 08:00:01.679170 10230 consensus_queue.cc:260] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:01.679462 10230 ts_tablet_manager.cc:1398] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:01.680307 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session 7a80bdedf813425faf7617b4b07c9f36-a4b3aaeeb72e47e08c938b47310dd402 received from {username='slave'} at 127.0.99.133:33299
I20240501 08:00:01.680400 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session 7a80bdedf813425faf7617b4b07c9f36-a4b3aaeeb72e47e08c938b47310dd402 on tablet a4b3aaeeb72e47e08c938b47310dd402 with peer 7a80bdedf813425faf7617b4b07c9f36
I20240501 08:00:01.688756 10236 tablet_bootstrap.cc:492] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:01.689052 10236 tablet_bootstrap.cc:492] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap complete.
I20240501 08:00:01.689234 10236 ts_tablet_manager.cc:1367] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8: Time spent bootstrapping tablet: real 0.023s user 0.023s sys 0.000s
I20240501 08:00:01.689719 10236 raft_consensus.cc:348] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:01.689828 10236 raft_consensus.cc:729] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06c63579c7234194b5f6ef457dd5e3f8, State: Initialized, Role: LEARNER
I20240501 08:00:01.689908 10236 consensus_queue.cc:260] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:01.690266 10236 ts_tablet_manager.cc:1398] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:01.690965 6614 tablet_copy_service.cc:342] P d2e209265a41425fba98c9d9b0551bd4: Request end of tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-2bbceded7f37434ca25b1068bdfc8ca1 received from {username='slave'} at 127.0.99.136:45667
I20240501 08:00:01.691097 6614 tablet_copy_service.cc:433] P d2e209265a41425fba98c9d9b0551bd4: ending tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-2bbceded7f37434ca25b1068bdfc8ca1 on tablet 2bbceded7f37434ca25b1068bdfc8ca1 with peer 06c63579c7234194b5f6ef457dd5e3f8
I20240501 08:00:01.717999 10276 tablet_copy_client.cc:523] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:01.719460 10276 tablet_bootstrap.cc:492] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap starting.
I20240501 08:00:01.738224 10276 tablet_bootstrap.cc:492] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:01.738528 10276 tablet_bootstrap.cc:492] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap complete.
I20240501 08:00:01.738709 10276 ts_tablet_manager.cc:1367] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36: Time spent bootstrapping tablet: real 0.019s user 0.018s sys 0.000s
I20240501 08:00:01.739152 10276 raft_consensus.cc:348] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:01.739282 10276 raft_consensus.cc:729] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a80bdedf813425faf7617b4b07c9f36, State: Initialized, Role: LEARNER
I20240501 08:00:01.739368 10276 consensus_queue.cc:260] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:01.739634 10276 ts_tablet_manager.cc:1398] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:01.740233 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session 7a80bdedf813425faf7617b4b07c9f36-33863ce53ec14aab9040f75e0360e0af received from {username='slave'} at 127.0.99.133:60211
I20240501 08:00:01.740324 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session 7a80bdedf813425faf7617b4b07c9f36-33863ce53ec14aab9040f75e0360e0af on tablet 33863ce53ec14aab9040f75e0360e0af with peer 7a80bdedf813425faf7617b4b07c9f36
I20240501 08:00:01.741464 8294 raft_consensus.cc:1203] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:01.753307 10311 tablet_copy_client.cc:523] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:01.755419 10311 tablet_bootstrap.cc:492] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap starting.
I20240501 08:00:01.762634 7899 raft_consensus.cc:1203] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:01.777848 10311 tablet_bootstrap.cc:492] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:01.778234 10311 tablet_bootstrap.cc:492] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap complete.
I20240501 08:00:01.778445 10311 ts_tablet_manager.cc:1367] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: Time spent bootstrapping tablet: real 0.023s user 0.016s sys 0.006s
I20240501 08:00:01.778929 10311 raft_consensus.cc:348] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:01.779080 10311 raft_consensus.cc:729] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Initialized, Role: LEARNER
I20240501 08:00:01.779182 10311 consensus_queue.cc:260] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:01.779481 10311 ts_tablet_manager.cc:1398] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:01.780645 6614 tablet_copy_service.cc:342] P d2e209265a41425fba98c9d9b0551bd4: Request end of tablet copy session ef83010b77b441ffac2e41468a0a1bd1-96386dd2c62e45859e1f3b30510e8969 received from {username='slave'} at 127.0.99.135:54555
I20240501 08:00:01.780721 6614 tablet_copy_service.cc:433] P d2e209265a41425fba98c9d9b0551bd4: ending tablet copy session ef83010b77b441ffac2e41468a0a1bd1-96386dd2c62e45859e1f3b30510e8969 on tablet 96386dd2c62e45859e1f3b30510e8969 with peer ef83010b77b441ffac2e41468a0a1bd1
I20240501 08:00:01.801844 10353 tablet_copy_client.cc:523] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:01.803300 10353 tablet_bootstrap.cc:492] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce: Bootstrap starting.
I20240501 08:00:01.824517 10353 tablet_bootstrap.cc:492] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:01.824837 10353 tablet_bootstrap.cc:492] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce: Bootstrap complete.
I20240501 08:00:01.825040 10353 ts_tablet_manager.cc:1367] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce: Time spent bootstrapping tablet: real 0.022s user 0.018s sys 0.004s
I20240501 08:00:01.825486 10353 raft_consensus.cc:348] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:01.825613 10353 raft_consensus.cc:729] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 29afadb287c049d9bb777d674b3697ce, State: Initialized, Role: LEARNER
I20240501 08:00:01.825711 10353 consensus_queue.cc:260] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:01.826038 10353 ts_tablet_manager.cc:1398] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:01.826956 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session 29afadb287c049d9bb777d674b3697ce-98d43e2703b14f9ebf926b5f2e2b2c56 received from {username='slave'} at 127.0.99.134:36071
I20240501 08:00:01.827035 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session 29afadb287c049d9bb777d674b3697ce-98d43e2703b14f9ebf926b5f2e2b2c56 on tablet 98d43e2703b14f9ebf926b5f2e2b2c56 with peer 29afadb287c049d9bb777d674b3697ce
I20240501 08:00:01.920873 10388 tablet_copy_client.cc:523] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:01.922283 10388 tablet_bootstrap.cc:492] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce: Bootstrap starting.
I20240501 08:00:01.942134 10388 tablet_bootstrap.cc:492] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:01.942493 10388 tablet_bootstrap.cc:492] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce: Bootstrap complete.
I20240501 08:00:01.942729 10388 ts_tablet_manager.cc:1367] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce: Time spent bootstrapping tablet: real 0.021s user 0.019s sys 0.000s
I20240501 08:00:01.943172 10388 raft_consensus.cc:348] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:01.943292 10388 raft_consensus.cc:729] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 29afadb287c049d9bb777d674b3697ce, State: Initialized, Role: LEARNER
I20240501 08:00:01.943374 10388 consensus_queue.cc:260] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:01.943634 10388 ts_tablet_manager.cc:1398] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:01.944288 6614 tablet_copy_service.cc:342] P d2e209265a41425fba98c9d9b0551bd4: Request end of tablet copy session 29afadb287c049d9bb777d674b3697ce-4e052973ae7744e8887d3df0d420f411 received from {username='slave'} at 127.0.99.134:43243
I20240501 08:00:01.944375 6614 tablet_copy_service.cc:433] P d2e209265a41425fba98c9d9b0551bd4: ending tablet copy session 29afadb287c049d9bb777d674b3697ce-4e052973ae7744e8887d3df0d420f411 on tablet 4e052973ae7744e8887d3df0d420f411 with peer 29afadb287c049d9bb777d674b3697ce
I20240501 08:00:01.945713 10393 tablet_copy_client.cc:523] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:01.947286 10393 tablet_bootstrap.cc:492] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap starting.
I20240501 08:00:01.947422 8031 raft_consensus.cc:1203] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:01.967579 10393 tablet_bootstrap.cc:492] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:01.967887 10393 tablet_bootstrap.cc:492] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap complete.
I20240501 08:00:01.968106 10393 ts_tablet_manager.cc:1367] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8: Time spent bootstrapping tablet: real 0.021s user 0.018s sys 0.004s
I20240501 08:00:01.968528 10393 raft_consensus.cc:348] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:01.968642 10393 raft_consensus.cc:729] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06c63579c7234194b5f6ef457dd5e3f8, State: Initialized, Role: LEARNER
I20240501 08:00:01.968721 10393 consensus_queue.cc:260] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:01.968964 10393 ts_tablet_manager.cc:1398] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:01.969612 6614 tablet_copy_service.cc:342] P d2e209265a41425fba98c9d9b0551bd4: Request end of tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-dbc4b418e4c5493fa8a7734740225200 received from {username='slave'} at 127.0.99.136:45667
I20240501 08:00:01.969687 6614 tablet_copy_service.cc:433] P d2e209265a41425fba98c9d9b0551bd4: ending tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-dbc4b418e4c5493fa8a7734740225200 on tablet dbc4b418e4c5493fa8a7734740225200 with peer 06c63579c7234194b5f6ef457dd5e3f8
I20240501 08:00:01.977910 8294 raft_consensus.cc:1203] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:02.008950 10398 tablet_copy_client.cc:523] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:02.010279 10398 tablet_bootstrap.cc:492] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce: Bootstrap starting.
I20240501 08:00:02.027700 10403 tablet_copy_client.cc:523] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:02.028903 10398 tablet_bootstrap.cc:492] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:02.029165 10398 tablet_bootstrap.cc:492] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce: Bootstrap complete.
I20240501 08:00:02.029346 10398 ts_tablet_manager.cc:1367] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce: Time spent bootstrapping tablet: real 0.019s user 0.019s sys 0.000s
I20240501 08:00:02.029459 10403 tablet_bootstrap.cc:492] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap starting.
I20240501 08:00:02.029839 10398 raft_consensus.cc:348] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:02.029979 10398 raft_consensus.cc:729] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 29afadb287c049d9bb777d674b3697ce, State: Initialized, Role: LEARNER
I20240501 08:00:02.030081 10398 consensus_queue.cc:260] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:02.030359 10398 ts_tablet_manager.cc:1398] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:02.031956 6220 tablet_copy_service.cc:342] P 953b945e3e7e4c71a61aa99c1208ab1d: Request end of tablet copy session 29afadb287c049d9bb777d674b3697ce-a97136a7017541eb8e4afb281c97e3cb received from {username='slave'} at 127.0.99.134:42931
I20240501 08:00:02.032063 6220 tablet_copy_service.cc:433] P 953b945e3e7e4c71a61aa99c1208ab1d: ending tablet copy session 29afadb287c049d9bb777d674b3697ce-a97136a7017541eb8e4afb281c97e3cb on tablet a97136a7017541eb8e4afb281c97e3cb with peer 29afadb287c049d9bb777d674b3697ce
I20240501 08:00:02.051398 10403 tablet_bootstrap.cc:492] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:02.051678 10403 tablet_bootstrap.cc:492] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap complete.
I20240501 08:00:02.051852 10403 ts_tablet_manager.cc:1367] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36: Time spent bootstrapping tablet: real 0.022s user 0.020s sys 0.002s
I20240501 08:00:02.052342 10403 raft_consensus.cc:348] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:02.052485 10403 raft_consensus.cc:729] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a80bdedf813425faf7617b4b07c9f36, State: Initialized, Role: LEARNER
I20240501 08:00:02.052585 10403 consensus_queue.cc:260] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:02.052963 10403 ts_tablet_manager.cc:1398] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 08:00:02.053663 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session 7a80bdedf813425faf7617b4b07c9f36-5c411a11ec484cd5bd063e5fc3868e29 received from {username='slave'} at 127.0.99.133:60211
I20240501 08:00:02.053750 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session 7a80bdedf813425faf7617b4b07c9f36-5c411a11ec484cd5bd063e5fc3868e29 on tablet 5c411a11ec484cd5bd063e5fc3868e29 with peer 7a80bdedf813425faf7617b4b07c9f36
I20240501 08:00:02.072154 7899 raft_consensus.cc:1203] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:02.077548 8162 raft_consensus.cc:1203] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:02.091485 10443 tablet_copy_client.cc:523] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:02.093164 10443 tablet_bootstrap.cc:492] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce: Bootstrap starting.
I20240501 08:00:02.118239 10443 tablet_bootstrap.cc:492] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:02.118590 10443 tablet_bootstrap.cc:492] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce: Bootstrap complete.
I20240501 08:00:02.118809 10443 ts_tablet_manager.cc:1367] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce: Time spent bootstrapping tablet: real 0.026s user 0.017s sys 0.005s
I20240501 08:00:02.119254 10443 raft_consensus.cc:348] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:02.119369 10443 raft_consensus.cc:729] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 29afadb287c049d9bb777d674b3697ce, State: Initialized, Role: LEARNER
I20240501 08:00:02.119442 10443 consensus_queue.cc:260] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:02.119879 10443 ts_tablet_manager.cc:1398] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:02.120582 6614 tablet_copy_service.cc:342] P d2e209265a41425fba98c9d9b0551bd4: Request end of tablet copy session 29afadb287c049d9bb777d674b3697ce-6bf00ed2af0e42ca94c99a701fbe7dd1 received from {username='slave'} at 127.0.99.134:43243
I20240501 08:00:02.120682 6614 tablet_copy_service.cc:433] P d2e209265a41425fba98c9d9b0551bd4: ending tablet copy session 29afadb287c049d9bb777d674b3697ce-6bf00ed2af0e42ca94c99a701fbe7dd1 on tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 with peer 29afadb287c049d9bb777d674b3697ce
I20240501 08:00:02.145040 8031 raft_consensus.cc:1203] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:02.177845 8031 raft_consensus.cc:1203] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:02.193701 7899 raft_consensus.cc:1203] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:02.227571 8162 raft_consensus.cc:1203] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:02.313423 9800 raft_consensus.cc:1050] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 to VOTER
I20240501 08:00:02.313897 9800 consensus_queue.cc:237] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:02.315318 6593 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.315318 7899 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.315470 6329 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.315730 10858 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.315956 9800 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.316169 10856 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.318125 9800 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.318588 6593 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.318584 7899 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.320461 6329 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.321149 6053 catalog_manager.cc:5484] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 reported cstate change: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.326257 6462 consensus_queue.cc:237] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:02.327456 6329 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.327486 7899 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.327872 10839 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.328207 10856 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.330505 10839 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.331418 7899 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.333088 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 33863ce53ec14aab9040f75e0360e0af with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:02.333657 6051 catalog_manager.cc:5484] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:02.334365 6329 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.337249 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet 33863ce53ec14aab9040f75e0360e0af with delete_type TABLET_DATA_TOMBSTONED (TS d2e209265a41425fba98c9d9b0551bd4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60522
I20240501 08:00:02.337594 10875 tablet_replica.cc:330] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4: stopping tablet replica
I20240501 08:00:02.337738 10875 raft_consensus.cc:2229] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:02.337909 10875 raft_consensus.cc:2258] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:02.338799 10875 ts_tablet_manager.cc:1875] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:02.341148 10875 ts_tablet_manager.cc:1888] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:02.341267 10875 log.cc:1196] T 33863ce53ec14aab9040f75e0360e0af P d2e209265a41425fba98c9d9b0551bd4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/33863ce53ec14aab9040f75e0360e0af
I20240501 08:00:02.341770 6046 catalog_manager.cc:4830] TS d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501): tablet 33863ce53ec14aab9040f75e0360e0af (table rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]) successfully deleted
I20240501 08:00:02.360453 10892 raft_consensus.cc:1050] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4: attempting to promote NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 to VOTER
I20240501 08:00:02.360924 10892 consensus_queue.cc:237] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:02.362515 6329 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.362653 6201 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.362879 8294 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.363087 10895 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.363405 10893 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.363608 10892 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.366245 10895 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.368381 8294 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.368484 6329 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.369046 6201 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.371270 6057 catalog_manager.cc:5484] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:02.378115 6593 consensus_queue.cc:237] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:02.379786 6201 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.379835 8294 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.380580 10892 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.380864 10895 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.382989 10895 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.383476 6201 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.385200 8294 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.385963 6055 catalog_manager.cc:5484] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:02.387197 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2bbceded7f37434ca25b1068bdfc8ca1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:02.389597 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bbceded7f37434ca25b1068bdfc8ca1 with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60224
I20240501 08:00:02.390467 10914 tablet_replica.cc:330] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 08:00:02.390661 10914 raft_consensus.cc:2229] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:02.390882 10914 raft_consensus.cc:2258] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:02.391671 10914 ts_tablet_manager.cc:1875] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:02.395854 10914 ts_tablet_manager.cc:1888] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:02.396044 10914 log.cc:1196] T 2bbceded7f37434ca25b1068bdfc8ca1 P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/2bbceded7f37434ca25b1068bdfc8ca1
I20240501 08:00:02.396569 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet 2bbceded7f37434ca25b1068bdfc8ca1 (table rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]) successfully deleted
I20240501 08:00:02.431797 10893 raft_consensus.cc:1050] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4: attempting to promote NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 to VOTER
I20240501 08:00:02.432315 10893 consensus_queue.cc:237] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:02.434127 6201 raft_consensus.cc:1261] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.434535 6462 raft_consensus.cc:1261] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.434558 8294 raft_consensus.cc:1261] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.434623 9900 consensus_queue.cc:1035] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.435004 10893 consensus_queue.cc:1035] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.435480 9900 consensus_queue.cc:1035] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.437892 10892 raft_consensus.cc:2932] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.438346 6462 raft_consensus.cc:2932] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.440097 8294 raft_consensus.cc:2932] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.441268 6051 catalog_manager.cc:5484] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:02.441267 6201 raft_consensus.cc:2932] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.447175 6593 consensus_queue.cc:237] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:02.448485 8294 raft_consensus.cc:1261] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.448503 6201 raft_consensus.cc:1261] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.448956 10892 consensus_queue.cc:1035] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.449404 10892 consensus_queue.cc:1035] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.450917 9900 raft_consensus.cc:2932] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.451426 6198 raft_consensus.cc:2932] T dbc4b418e4c5493fa8a7734740225200 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.452499 8294 raft_consensus.cc:2932] T dbc4b418e4c5493fa8a7734740225200 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:02.454509 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet dbc4b418e4c5493fa8a7734740225200 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:02.454768 6055 catalog_manager.cc:5484] T dbc4b418e4c5493fa8a7734740225200 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:02.457995 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet dbc4b418e4c5493fa8a7734740225200 with delete_type TABLET_DATA_TOMBSTONED (TS 5db2970f4bbd4a3c92d7d913f99716b1 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:51996
I20240501 08:00:02.458434 10954 tablet_replica.cc:330] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 08:00:02.458636 10954 raft_consensus.cc:2229] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:02.458896 10954 raft_consensus.cc:2258] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:02.459769 10954 ts_tablet_manager.cc:1875] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:02.462116 10954 ts_tablet_manager.cc:1888] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:02.462249 10954 log.cc:1196] T dbc4b418e4c5493fa8a7734740225200 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/dbc4b418e4c5493fa8a7734740225200
I20240501 08:00:02.462798 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet dbc4b418e4c5493fa8a7734740225200 (table rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]) successfully deleted
I20240501 08:00:02.535897 7817 raft_consensus.cc:1050] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 to VOTER
I20240501 08:00:02.536379 7817 consensus_queue.cc:237] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:02.537761 8162 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.538019 6201 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.538594 6593 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.539065 11013 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.539989 10858 raft_consensus.cc:1050] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER 29afadb287c049d9bb777d674b3697ce to VOTER
I20240501 08:00:02.540419 10858 consensus_queue.cc:237] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:02.540598 11010 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.541805 8031 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.541945 11010 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.542132 6329 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.542307 10839 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.543181 10856 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.543779 6201 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.544196 10856 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.545336 11010 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:02.545985 10856 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.546283 6201 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:02.547371 6593 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:02.547819 8162 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:02.549158 6051 catalog_manager.cc:5484] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:02.549335 8031 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.552592 6057 catalog_manager.cc:5484] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce reported cstate change: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.553710 6329 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.554236 6201 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.560592 6462 consensus_queue.cc:237] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:02.562072 6329 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.561956 8031 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.562741 9800 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.563102 10839 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.565261 9800 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.565904 8031 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.567667 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 98d43e2703b14f9ebf926b5f2e2b2c56 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:02.568301 6331 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.568873 6052 catalog_manager.cc:5484] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce reported cstate change: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.572060 6181 tablet_service.cc:1514] Processing DeleteTablet for tablet 98d43e2703b14f9ebf926b5f2e2b2c56 with delete_type TABLET_DATA_TOMBSTONED (TS 953b945e3e7e4c71a61aa99c1208ab1d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45448
I20240501 08:00:02.573565 6331 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92"
dest_uuid: "4a155eaa444d440d90fde420951458e4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49654
I20240501 08:00:02.573761 6331 raft_consensus.cc:595] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:02.574340 7817 raft_consensus.cc:1050] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 to VOTER
I20240501 08:00:02.574748 7817 consensus_queue.cc:237] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:02.576424 6462 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.576411 6593 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.576779 7899 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.576918 11010 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.577026 11036 tablet_replica.cc:330] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d: stopping tablet replica
I20240501 08:00:02.577173 11036 raft_consensus.cc:2229] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:02.577430 11010 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.577711 11012 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.579488 11036 raft_consensus.cc:2258] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:02.580646 11013 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.580973 7899 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.581223 6462 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.584445 6055 catalog_manager.cc:5484] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.586064 11036 ts_tablet_manager.cc:1875] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:02.588558 11036 ts_tablet_manager.cc:1888] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:02.588685 11036 log.cc:1196] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/98d43e2703b14f9ebf926b5f2e2b2c56
I20240501 08:00:02.588824 6593 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.589303 6046 catalog_manager.cc:4830] TS 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189): tablet 98d43e2703b14f9ebf926b5f2e2b2c56 (table rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]) successfully deleted
I20240501 08:00:02.591914 6331 consensus_queue.cc:237] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:02.593222 6593 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.593297 7899 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.593742 11012 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.593948 7817 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.595582 11010 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.596071 6593 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.596119 7899 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.597656 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a4b3aaeeb72e47e08c938b47310dd402 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:02.598627 6052 catalog_manager.cc:5484] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 reported cstate change: config changed from index 5 to 6, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.601419 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b3aaeeb72e47e08c938b47310dd402 with delete_type TABLET_DATA_TOMBSTONED (TS 5db2970f4bbd4a3c92d7d913f99716b1 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:51996
I20240501 08:00:02.601965 10954 tablet_replica.cc:330] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 08:00:02.602134 10954 raft_consensus.cc:2229] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:02.602291 10954 raft_consensus.cc:2258] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:02.603160 10954 ts_tablet_manager.cc:1875] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:02.605343 10954 ts_tablet_manager.cc:1888] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:02.605432 10954 log.cc:1196] T a4b3aaeeb72e47e08c938b47310dd402 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/a4b3aaeeb72e47e08c938b47310dd402
I20240501 08:00:02.605943 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet a4b3aaeeb72e47e08c938b47310dd402 (table rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]) successfully deleted
I20240501 08:00:02.631239 9800 raft_consensus.cc:1050] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 to VOTER
I20240501 08:00:02.631734 9800 consensus_queue.cc:237] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:02.633249 6201 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.633296 6593 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.633384 7899 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.633697 10839 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.633886 9800 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.634088 10856 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.636291 10839 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.636768 7899 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.636917 6201 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.639492 6054 catalog_manager.cc:5484] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 reported cstate change: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.640192 6593 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.643122 6462 consensus_queue.cc:237] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:02.644731 7899 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.645066 6201 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.645267 9800 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.645514 10856 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.646837 8031 raft_consensus.cc:1203] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:02.647133 9800 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.647986 7899 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.648929 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5c411a11ec484cd5bd063e5fc3868e29 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:02.649260 6201 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:02.649794 6055 catalog_manager.cc:5484] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 5 to 6, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:02.653602 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c411a11ec484cd5bd063e5fc3868e29 with delete_type TABLET_DATA_TOMBSTONED (TS d2e209265a41425fba98c9d9b0551bd4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60522
I20240501 08:00:02.653885 10875 tablet_replica.cc:330] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4: stopping tablet replica
I20240501 08:00:02.654076 10875 raft_consensus.cc:2229] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:02.654217 10875 raft_consensus.cc:2258] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:02.654928 10875 ts_tablet_manager.cc:1875] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:02.656807 10875 ts_tablet_manager.cc:1888] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:02.656881 10875 log.cc:1196] T 5c411a11ec484cd5bd063e5fc3868e29 P d2e209265a41425fba98c9d9b0551bd4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/5c411a11ec484cd5bd063e5fc3868e29
I20240501 08:00:02.657367 6046 catalog_manager.cc:4830] TS d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501): tablet 5c411a11ec484cd5bd063e5fc3868e29 (table rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]) successfully deleted
I20240501 08:00:02.672550 10895 raft_consensus.cc:1050] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: attempting to promote NON_VOTER 29afadb287c049d9bb777d674b3697ce to VOTER
I20240501 08:00:02.672945 10895 consensus_queue.cc:237] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:02.674417 6331 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.674479 6201 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.674557 8031 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.674914 10893 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.675189 9900 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.675560 10892 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.678081 10892 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.678563 6331 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.679844 6198 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.680814 8031 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.681224 6055 catalog_manager.cc:5484] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.685122 6593 consensus_queue.cc:237] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:02.686260 8031 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.686412 6331 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.686697 10895 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.686900 9900 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.688364 10895 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.688854 8031 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.689007 6331 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.690143 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4e052973ae7744e8887d3df0d420f411 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:02.691272 6053 catalog_manager.cc:5484] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce reported cstate change: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.693850 6181 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e052973ae7744e8887d3df0d420f411 with delete_type TABLET_DATA_TOMBSTONED (TS 953b945e3e7e4c71a61aa99c1208ab1d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45448
I20240501 08:00:02.694115 11036 tablet_replica.cc:330] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d: stopping tablet replica
I20240501 08:00:02.694260 11036 raft_consensus.cc:2229] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:02.694414 11036 raft_consensus.cc:2258] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:02.695184 11036 ts_tablet_manager.cc:1875] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:02.697026 11036 ts_tablet_manager.cc:1888] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:02.697106 11036 log.cc:1196] T 4e052973ae7744e8887d3df0d420f411 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/4e052973ae7744e8887d3df0d420f411
I20240501 08:00:02.697755 6046 catalog_manager.cc:4830] TS 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189): tablet 4e052973ae7744e8887d3df0d420f411 (table rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]) successfully deleted
I20240501 08:00:02.721321 10892 raft_consensus.cc:1050] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4: attempting to promote NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 to VOTER
I20240501 08:00:02.721853 10892 consensus_queue.cc:237] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:02.723397 8162 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.723387 6331 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.723790 10892 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.723367 6201 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.724045 9900 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.724392 9900 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.727170 10895 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:02.727464 6329 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:02.728691 6201 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:02.729193 8162 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:02.730607 6054 catalog_manager.cc:5484] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 4 to 5, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:02.766083 11058 raft_consensus.cc:1050] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d: attempting to promote NON_VOTER 29afadb287c049d9bb777d674b3697ce to VOTER
I20240501 08:00:02.766618 11058 consensus_queue.cc:237] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:02.768234 8031 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.768235 6462 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.768266 6329 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:02.768661 11058 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.768899 11061 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.769091 11060 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:02.771211 11061 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.771734 6329 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.772341 6462 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.774755 6053 catalog_manager.cc:5484] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:02.774816 8031 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.778280 6201 consensus_queue.cc:237] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:02.779460 6462 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.779460 8031 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:02.780025 10730 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.780270 11060 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:02.782200 11058 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.782660 8031 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.782670 6462 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.785418 6052 catalog_manager.cc:5484] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce reported cstate change: config changed from index 5 to 6, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:02.785598 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a97136a7017541eb8e4afb281c97e3cb with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:02.788341 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet a97136a7017541eb8e4afb281c97e3cb with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:60224
I20240501 08:00:02.788592 10914 tablet_replica.cc:330] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 08:00:02.788734 10914 raft_consensus.cc:2229] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:02.788901 10914 raft_consensus.cc:2258] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:02.789703 10914 ts_tablet_manager.cc:1875] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:02.791853 10914 ts_tablet_manager.cc:1888] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:02.791947 10914 log.cc:1196] T a97136a7017541eb8e4afb281c97e3cb P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/a97136a7017541eb8e4afb281c97e3cb
I20240501 08:00:02.792477 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet a97136a7017541eb8e4afb281c97e3cb (table rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]) successfully deleted
I20240501 08:00:02.838361 6329 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92"
dest_uuid: "4a155eaa444d440d90fde420951458e4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49682
I20240501 08:00:02.838514 6329 raft_consensus.cc:595] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:02.849469 6593 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96386dd2c62e45859e1f3b30510e8969"
dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43900
I20240501 08:00:02.849704 6593 raft_consensus.cc:595] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:02.866688 6329 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92"
dest_uuid: "4a155eaa444d440d90fde420951458e4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49698
I20240501 08:00:02.866856 6329 raft_consensus.cc:595] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:02.878849 6593 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96386dd2c62e45859e1f3b30510e8969"
dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43932
I20240501 08:00:02.879009 6593 raft_consensus.cc:595] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:02.925866 7817 raft_consensus.cc:979] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4: : Instructing follower d2e209265a41425fba98c9d9b0551bd4 to start an election
I20240501 08:00:02.925992 7817 raft_consensus.cc:1067] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Signalling peer d2e209265a41425fba98c9d9b0551bd4 to start an election
I20240501 08:00:02.926522 6593 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92"
dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
from {username='slave'} at 127.0.99.130:48525
I20240501 08:00:02.926676 6593 raft_consensus.cc:482] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:02.926738 6593 raft_consensus.cc:3037] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:02.927608 6593 raft_consensus.cc:504] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:02.928040 6593 leader_election.cc:288] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 2 election: Requested vote from peers 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937)
I20240501 08:00:02.928527 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 08:00:02.928584 8162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "ef83010b77b441ffac2e41468a0a1bd1"
I20240501 08:00:02.928673 6201 raft_consensus.cc:3037] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:02.928620 6329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92" candidate_uuid: "d2e209265a41425fba98c9d9b0551bd4" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 08:00:02.928758 6329 raft_consensus.cc:3032] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:00:02.928754 8162 raft_consensus.cc:3037] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:02.928802 6329 raft_consensus.cc:729] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Running, Role: LEADER
I20240501 08:00:02.928972 6329 consensus_queue.cc:260] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:02.929152 6329 raft_consensus.cc:3037] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:02.929863 6201 raft_consensus.cc:2454] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 2.
I20240501 08:00:02.929869 8162 raft_consensus.cc:2454] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 2.
I20240501 08:00:02.929915 6329 raft_consensus.cc:2454] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2e209265a41425fba98c9d9b0551bd4 in term 2.
I20240501 08:00:02.930263 6528 leader_election.cc:302] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4, ef83010b77b441ffac2e41468a0a1bd1; no voters:
I20240501 08:00:02.930457 9900 raft_consensus.cc:2781] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:00:02.930567 9900 raft_consensus.cc:686] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 2 LEADER]: Becoming Leader. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Running, Role: LEADER
I20240501 08:00:02.930716 9900 consensus_queue.cc:237] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:02.933027 6055 catalog_manager.cc:5484] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: term changed from 1 to 2, leader changed from 4a155eaa444d440d90fde420951458e4 (127.0.99.130) to d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:03.024603 10892 raft_consensus.cc:1050] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4: attempting to promote NON_VOTER 29afadb287c049d9bb777d674b3697ce to VOTER
I20240501 08:00:03.025128 10892 consensus_queue.cc:237] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:03.026563 6201 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:03.026575 8031 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:03.026621 6462 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:03.027000 9900 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:03.027294 10895 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:03.027491 10893 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:03.029491 9900 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:03.030023 6201 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:03.030108 6462 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:03.032528 8031 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:03.032708 6055 catalog_manager.cc:5484] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 4 to 5, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:03.035704 6593 consensus_queue.cc:237] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:03.036880 8031 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:03.036911 6462 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:03.037246 10895 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:03.037438 9900 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:03.038911 10892 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:03.039322 6462 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:03.040287 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:03.040668 8031 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:03.042349 6052 catalog_manager.cc:5484] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 5 to 6, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:03.044603 6181 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 with delete_type TABLET_DATA_TOMBSTONED (TS 953b945e3e7e4c71a61aa99c1208ab1d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45448
I20240501 08:00:03.044855 11036 tablet_replica.cc:330] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d: stopping tablet replica
I20240501 08:00:03.045008 11036 raft_consensus.cc:2229] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:03.045181 11036 raft_consensus.cc:2258] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:03.046131 11036 ts_tablet_manager.cc:1875] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:03.048013 11036 ts_tablet_manager.cc:1888] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:03.048089 11036 log.cc:1196] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/6bf00ed2af0e42ca94c99a701fbe7dd1
I20240501 08:00:03.048609 6046 catalog_manager.cc:4830] TS 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189): tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 (table rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]) successfully deleted
I20240501 08:00:03.104187 6593 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96386dd2c62e45859e1f3b30510e8969"
dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43968
I20240501 08:00:03.104332 6593 raft_consensus.cc:595] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:03.121562 6593 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "96386dd2c62e45859e1f3b30510e8969"
dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43994
I20240501 08:00:03.121727 6593 raft_consensus.cc:595] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:03.271517 10895 raft_consensus.cc:979] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4: : Instructing follower ef83010b77b441ffac2e41468a0a1bd1 to start an election
I20240501 08:00:03.271641 10895 raft_consensus.cc:1067] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Signalling peer ef83010b77b441ffac2e41468a0a1bd1 to start an election
I20240501 08:00:03.272254 8162 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "96386dd2c62e45859e1f3b30510e8969"
dest_uuid: "ef83010b77b441ffac2e41468a0a1bd1"
from {username='slave'} at 127.0.99.132:53569
I20240501 08:00:03.272437 8162 raft_consensus.cc:482] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:03.272500 8162 raft_consensus.cc:3037] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:03.273325 8162 raft_consensus.cc:504] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:03.274619 8162 leader_election.cc:288] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [CANDIDATE]: Term 2 election: Requested vote from peers 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 08:00:03.274960 6201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96386dd2c62e45859e1f3b30510e8969" candidate_uuid: "ef83010b77b441ffac2e41468a0a1bd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 08:00:03.275095 6201 raft_consensus.cc:3037] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:03.275118 6329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96386dd2c62e45859e1f3b30510e8969" candidate_uuid: "ef83010b77b441ffac2e41468a0a1bd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 08:00:03.275245 6329 raft_consensus.cc:3037] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:03.275784 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96386dd2c62e45859e1f3b30510e8969" candidate_uuid: "ef83010b77b441ffac2e41468a0a1bd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 08:00:03.275903 6593 raft_consensus.cc:3032] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:00:03.275967 6593 raft_consensus.cc:729] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Running, Role: LEADER
I20240501 08:00:03.276113 6201 raft_consensus.cc:2454] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef83010b77b441ffac2e41468a0a1bd1 in term 2.
I20240501 08:00:03.276146 6593 consensus_queue.cc:260] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:03.276304 6329 raft_consensus.cc:2454] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef83010b77b441ffac2e41468a0a1bd1 in term 2.
I20240501 08:00:03.276360 6593 raft_consensus.cc:3037] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:03.276595 8098 leader_election.cc:302] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, 953b945e3e7e4c71a61aa99c1208ab1d, ef83010b77b441ffac2e41468a0a1bd1; no voters:
I20240501 08:00:03.276928 11179 raft_consensus.cc:2781] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:00:03.277051 11179 raft_consensus.cc:686] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 LEADER]: Becoming Leader. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Running, Role: LEADER
I20240501 08:00:03.277163 6593 raft_consensus.cc:2454] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef83010b77b441ffac2e41468a0a1bd1 in term 2.
I20240501 08:00:03.277338 11179 consensus_queue.cc:237] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:03.280017 6054 catalog_manager.cc:5484] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: term changed from 1 to 2, leader changed from d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) to ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135). New cstate: current_term: 2 leader_uuid: "ef83010b77b441ffac2e41468a0a1bd1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:03.334815 6329 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 08:00:03.335436 10895 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:03.472023 8162 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 08:00:03.472446 10895 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:03.475695 6201 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 08:00:03.476764 10893 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:03.671746 6593 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 08:00:03.672283 11179 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:03.831085 6329 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 08:00:03.831563 11179 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:03.834511 6201 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 08:00:03.834932 11219 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:04.026470 6593 consensus_queue.cc:237] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:04.027587 8162 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 08:00:04.027653 6201 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 08:00:04.028060 11233 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.028239 11232 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.029762 11234 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:04.030180 8162 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:04.030144 6198 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:04.032457 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 71bcfcbbdfc0491ea936168dd699fc92 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:04.032380 6054 catalog_manager.cc:5484] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 5 to 7, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 2 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:04.035353 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet 71bcfcbbdfc0491ea936168dd699fc92 with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:60224
I20240501 08:00:04.035720 11238 tablet_replica.cc:330] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 08:00:04.035881 11238 raft_consensus.cc:2229] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:04.036047 11238 raft_consensus.cc:2258] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:04.036796 11238 ts_tablet_manager.cc:1875] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:04.038882 11238 ts_tablet_manager.cc:1888] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240501 08:00:04.038947 11238 log.cc:1196] T 71bcfcbbdfc0491ea936168dd699fc92 P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/71bcfcbbdfc0491ea936168dd699fc92
I20240501 08:00:04.039450 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet 71bcfcbbdfc0491ea936168dd699fc92 (table rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]) successfully deleted
I20240501 08:00:04.369145 8162 consensus_queue.cc:237] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:04.370645 6198 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 08:00:04.370674 6329 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 08:00:04.371104 11179 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.371320 11254 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.373152 11179 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:04.374853 6044 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 96386dd2c62e45859e1f3b30510e8969 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:04.375758 6198 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:04.375973 6053 catalog_manager.cc:5484] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: config changed from index 5 to 7, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New cstate: current_term: 2 leader_uuid: "ef83010b77b441ffac2e41468a0a1bd1" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:04.376278 6329 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:04.378429 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet 96386dd2c62e45859e1f3b30510e8969 with delete_type TABLET_DATA_TOMBSTONED (TS d2e209265a41425fba98c9d9b0551bd4 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:60522
I20240501 08:00:04.378809 11257 tablet_replica.cc:330] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4: stopping tablet replica
I20240501 08:00:04.379421 11257 raft_consensus.cc:2229] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:04.379539 11257 raft_consensus.cc:2258] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:04.380290 11257 ts_tablet_manager.cc:1875] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:04.382438 11257 ts_tablet_manager.cc:1888] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240501 08:00:04.382513 11257 log.cc:1196] T 96386dd2c62e45859e1f3b30510e8969 P d2e209265a41425fba98c9d9b0551bd4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/96386dd2c62e45859e1f3b30510e8969
I20240501 08:00:04.383029 6046 catalog_manager.cc:4830] TS d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501): tablet 96386dd2c62e45859e1f3b30510e8969 (table rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]) successfully deleted
I20240501 08:00:04.475919 8274 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:04.476964 8011 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:04.486351 6441 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 08:00:04.487412 6572 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:04.489257 7880 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 08:00:04.496366 8142 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 08:00:04.496770 6181 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:04.498868 9169 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:04.527558 6311 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 08:00:04.562783 6462 consensus_queue.cc:237] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:04.564440 6329 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.564589 8294 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.564940 11315 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.565138 10839 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.567235 11217 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.567737 8294 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
W20240501 08:00:04.568737 6396 consensus_peers.cc:485] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389): Couldn't send request to peer 773008dcb81842c3bdf9ce259bb66b8a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e60dad8ddd2402faae35be18662c8c3. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.570140 6051 catalog_manager.cc:5484] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:04.570997 6329 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.579326 6593 consensus_queue.cc:237] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:04.581183 6198 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240501 08:00:04.581614 11327 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:04.583470 10893 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 08:00:04.584239 6198 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 08:00:04.585002 8162 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240501 08:00:04.585929 10893 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:04.586798 6051 catalog_manager.cc:5484] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 7 to 8, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New cstate: current_term: 2 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 08:00:04.587513 8162 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
W20240501 08:00:04.595767 6528 consensus_peers.cc:485] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 -> Peer 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247): Couldn't send request to peer 7a80bdedf813425faf7617b4b07c9f36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 71bcfcbbdfc0491ea936168dd699fc92. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.601352 6462 consensus_queue.cc:237] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:04.602746 7899 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.602955 6198 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.603410 11315 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240501 08:00:04.603785 6398 consensus_peers.cc:485] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer 29afadb287c049d9bb777d674b3697ce (127.0.99.134:46293): Couldn't send request to peer 29afadb287c049d9bb777d674b3697ce. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5c411a11ec484cd5bd063e5fc3868e29. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.603782 11315 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.605504 11319 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 08:00:04.605892 7899 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 08:00:04.607550 6198 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 08:00:04.608444 6051 catalog_manager.cc:5484] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 6 to 7, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:04.617496 6462 consensus_queue.cc:237] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:04.619112 6329 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.619269 7899 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.619534 11315 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.619760 11319 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.621410 11319 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 08:00:04.622697 6329 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 08:00:04.623211 7899 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
W20240501 08:00:04.623416 6397 consensus_peers.cc:485] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937): Couldn't send request to peer ef83010b77b441ffac2e41468a0a1bd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33863ce53ec14aab9040f75e0360e0af. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.625545 6056 catalog_manager.cc:5484] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 6 to 7, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:04.631660 6462 consensus_queue.cc:237] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:04.633100 7899 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.633170 6593 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 08:00:04.633334 6396 consensus_peers.cc:485] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136:40449): Couldn't send request to peer 06c63579c7234194b5f6ef457dd5e3f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f24f629734649a3b31c7e293122d7ce. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.633572 10839 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.633913 10839 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.635432 10839 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 08:00:04.636157 7899 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 08:00:04.636240 6593 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 08:00:04.638193 6057 catalog_manager.cc:5484] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 6 to 7, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:04.646483 6329 consensus_queue.cc:237] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:04.648027 6593 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.648028 8162 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.648411 11012 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.648594 11359 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.650928 11359 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.651321 6593 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.651918 8162 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
W20240501 08:00:04.652139 6265 consensus_peers.cc:485] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 -> Peer 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389): Couldn't send request to peer 773008dcb81842c3bdf9ce259bb66b8a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f4c62996a5a64d978850f2f7a31009f4. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.654076 6051 catalog_manager.cc:5484] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.662402 6462 consensus_queue.cc:237] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:04.663779 6329 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.663851 8030 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 08:00:04.664053 6397 consensus_peers.cc:485] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 -> Peer 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247): Couldn't send request to peer 7a80bdedf813425faf7617b4b07c9f36. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98d43e2703b14f9ebf926b5f2e2b2c56. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.664237 11319 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.664443 11315 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.666361 11217 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 08:00:04.666985 6329 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 08:00:04.667093 8030 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } } }
I20240501 08:00:04.669327 6057 catalog_manager.cc:5484] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 6 to 7, NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:04.678179 6593 consensus_queue.cc:237] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:04.679901 6198 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.680080 8294 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.680356 11327 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.680711 11327 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.682221 11326 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.682868 6198 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.683985 8294 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.684813 6051 catalog_manager.cc:5484] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 08:00:04.686230 6527 consensus_peers.cc:485] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 -> Peer 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389): Couldn't send request to peer 773008dcb81842c3bdf9ce259bb66b8a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2bbceded7f37434ca25b1068bdfc8ca1. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.694330 6329 consensus_queue.cc:237] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:04.695848 6593 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.695920 8162 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 08:00:04.696190 6267 consensus_peers.cc:485] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 -> Peer 29afadb287c049d9bb777d674b3697ce (127.0.99.134:46293): Couldn't send request to peer 29afadb287c049d9bb777d674b3697ce. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de48f8c2870f4517b5349c30a51fce56. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.696317 11012 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.696514 11359 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.698284 11012 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 08:00:04.698838 8162 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 08:00:04.698985 6593 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } } }
I20240501 08:00:04.701417 6056 catalog_manager.cc:5484] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 6 to 7, NON_VOTER 29afadb287c049d9bb777d674b3697ce (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:04.709690 6593 consensus_queue.cc:237] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:04.711145 6329 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.711551 11326 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240501 08:00:04.712514 6528 consensus_peers.cc:485] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 -> Peer ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937): Couldn't send request to peer ef83010b77b441ffac2e41468a0a1bd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e052973ae7744e8887d3df0d420f411. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.713629 10893 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 08:00:04.715148 6329 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 08:00:04.716358 8030 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.716591 6057 catalog_manager.cc:5484] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 6 to 7, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:04.716782 11326 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.719276 8030 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 08:00:04.725836 6198 consensus_queue.cc:237] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:04.727531 6462 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.727604 8030 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 08:00:04.727802 6134 consensus_peers.cc:485] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d -> Peer 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136:40449): Couldn't send request to peer 06c63579c7234194b5f6ef457dd5e3f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a97136a7017541eb8e4afb281c97e3cb. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.728044 11404 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.728271 11403 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.730077 11192 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 08:00:04.730476 8030 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 08:00:04.730955 6462 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 08:00:04.733505 6051 catalog_manager.cc:5484] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce reported cstate change: config changed from index 6 to 7, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 08:00:04.742858 6198 consensus_queue.cc:237] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:04.744525 6462 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.744673 8162 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.745129 11403 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.745360 11192 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.747188 11192 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
W20240501 08:00:04.748386 6134 consensus_peers.cc:485] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d -> Peer 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389): Couldn't send request to peer 773008dcb81842c3bdf9ce259bb66b8a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16752f91ea894a7ba4017170c1dc6ec1. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.748828 8162 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.750878 6462 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.750936 6057 catalog_manager.cc:5484] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:04.758452 6329 consensus_queue.cc:237] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:04.759919 7899 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.759972 6593 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 08:00:04.760329 6266 consensus_peers.cc:485] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 -> Peer ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937): Couldn't send request to peer ef83010b77b441ffac2e41468a0a1bd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a4b3aaeeb72e47e08c938b47310dd402. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.760386 11360 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.760594 11359 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.762657 11012 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 08:00:04.763111 7899 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 08:00:04.763283 6593 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } } }
I20240501 08:00:04.765656 6056 catalog_manager.cc:5484] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 6 to 7, NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:04.775264 6329 consensus_queue.cc:237] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:04.776825 8162 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.776856 6593 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 08:00:04.777046 6265 consensus_peers.cc:485] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 -> Peer 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136:40449): Couldn't send request to peer 06c63579c7234194b5f6ef457dd5e3f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8bf587ac1d8141e99fceab28d89e4ea9. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.777298 11012 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.777500 11360 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.779291 11012 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 08:00:04.779711 8162 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 08:00:04.782522 6593 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 08:00:04.785193 6056 catalog_manager.cc:5484] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 6 to 7, NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } } }
I20240501 08:00:04.795363 6329 consensus_queue.cc:237] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:04.796906 8030 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:04.797061 6462 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 08:00:04.797159 6265 consensus_peers.cc:485] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 -> Peer 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389): Couldn't send request to peer 773008dcb81842c3bdf9ce259bb66b8a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41b9f1a284c34a9099cf1014db77f35c. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:04.797351 11360 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.797546 11359 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:04.799255 11012 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.799667 6462 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.799747 8030 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:04.802613 6056 catalog_manager.cc:5484] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:05.033794 11515 ts_tablet_manager.cc:902] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 08:00:05.034309 11515 tablet_copy_client.cc:308] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
I20240501 08:00:05.037977 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet 33863ce53ec14aab9040f75e0360e0af from peer ef83010b77b441ffac2e41468a0a1bd1 ({username='slave'} at 127.0.99.135:54715)
I20240501 08:00:05.038096 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet 33863ce53ec14aab9040f75e0360e0af from peer ef83010b77b441ffac2e41468a0a1bd1 at {username='slave'} at 127.0.99.135:54715: session id = ef83010b77b441ffac2e41468a0a1bd1-33863ce53ec14aab9040f75e0360e0af
I20240501 08:00:05.038854 6483 tablet_copy_source_session.cc:205] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.039674 11515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33863ce53ec14aab9040f75e0360e0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.041463 11515 tablet_copy_client.cc:783] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.041608 11515 tablet_copy_client.cc:647] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.042618 11519 tablet_copy_client.cc:1002] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.044821 11521 ts_tablet_manager.cc:902] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36: Initiating tablet copy from peer d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 08:00:05.045316 11521 tablet_copy_client.cc:308] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:42501
I20240501 08:00:05.045794 6614 tablet_copy_service.cc:140] P d2e209265a41425fba98c9d9b0551bd4: Received BeginTabletCopySession request for tablet 71bcfcbbdfc0491ea936168dd699fc92 from peer 7a80bdedf813425faf7617b4b07c9f36 ({username='slave'} at 127.0.99.133:52377)
I20240501 08:00:05.045915 6614 tablet_copy_service.cc:161] P d2e209265a41425fba98c9d9b0551bd4: Beginning new tablet copy session on tablet 71bcfcbbdfc0491ea936168dd699fc92 from peer 7a80bdedf813425faf7617b4b07c9f36 at {username='slave'} at 127.0.99.133:52377: session id = 7a80bdedf813425faf7617b4b07c9f36-71bcfcbbdfc0491ea936168dd699fc92
I20240501 08:00:05.046790 6614 tablet_copy_source_session.cc:205] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.047705 11521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71bcfcbbdfc0491ea936168dd699fc92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.049624 11521 tablet_copy_client.cc:783] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.049773 11521 tablet_copy_client.cc:647] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.050788 11525 tablet_copy_client.cc:1002] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.071586 11526 ts_tablet_manager.cc:902] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 08:00:05.072139 11526 tablet_copy_client.cc:308] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
I20240501 08:00:05.072638 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet 98d43e2703b14f9ebf926b5f2e2b2c56 from peer 7a80bdedf813425faf7617b4b07c9f36 ({username='slave'} at 127.0.99.133:60211)
I20240501 08:00:05.072742 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet 98d43e2703b14f9ebf926b5f2e2b2c56 from peer 7a80bdedf813425faf7617b4b07c9f36 at {username='slave'} at 127.0.99.133:60211: session id = 7a80bdedf813425faf7617b4b07c9f36-98d43e2703b14f9ebf926b5f2e2b2c56
I20240501 08:00:05.073450 6483 tablet_copy_source_session.cc:205] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.074285 11526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98d43e2703b14f9ebf926b5f2e2b2c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.076099 11526 tablet_copy_client.cc:783] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.076258 11526 tablet_copy_client.cc:647] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.077325 11530 tablet_copy_client.cc:1002] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.079609 11531 ts_tablet_manager.cc:902] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 08:00:05.080387 11531 tablet_copy_client.cc:308] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
I20240501 08:00:05.084316 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet 4e60dad8ddd2402faae35be18662c8c3 from peer 773008dcb81842c3bdf9ce259bb66b8a ({username='slave'} at 127.0.99.137:54799)
I20240501 08:00:05.084435 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet 4e60dad8ddd2402faae35be18662c8c3 from peer 773008dcb81842c3bdf9ce259bb66b8a at {username='slave'} at 127.0.99.137:54799: session id = 773008dcb81842c3bdf9ce259bb66b8a-4e60dad8ddd2402faae35be18662c8c3
I20240501 08:00:05.085346 6483 tablet_copy_source_session.cc:205] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.088845 11531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e60dad8ddd2402faae35be18662c8c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.094923 11531 tablet_copy_client.cc:783] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.095182 11531 tablet_copy_client.cc:647] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.096724 11535 tablet_copy_client.cc:1002] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.113750 11562 ts_tablet_manager.cc:902] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8: Initiating tablet copy from peer 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 08:00:05.114320 11562 tablet_copy_client.cc:308] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:44189
I20240501 08:00:05.114809 6220 tablet_copy_service.cc:140] P 953b945e3e7e4c71a61aa99c1208ab1d: Received BeginTabletCopySession request for tablet a97136a7017541eb8e4afb281c97e3cb from peer 06c63579c7234194b5f6ef457dd5e3f8 ({username='slave'} at 127.0.99.136:46025)
I20240501 08:00:05.114930 6220 tablet_copy_service.cc:161] P 953b945e3e7e4c71a61aa99c1208ab1d: Beginning new tablet copy session on tablet a97136a7017541eb8e4afb281c97e3cb from peer 06c63579c7234194b5f6ef457dd5e3f8 at {username='slave'} at 127.0.99.136:46025: session id = 06c63579c7234194b5f6ef457dd5e3f8-a97136a7017541eb8e4afb281c97e3cb
I20240501 08:00:05.115862 6220 tablet_copy_source_session.cc:205] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.116838 11562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97136a7017541eb8e4afb281c97e3cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.119422 11562 tablet_copy_client.cc:783] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.119657 11562 tablet_copy_client.cc:647] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.120698 11566 tablet_copy_client.cc:1002] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.136296 11582 ts_tablet_manager.cc:902] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 08:00:05.136821 11582 tablet_copy_client.cc:308] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
I20240501 08:00:05.137243 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet 5c411a11ec484cd5bd063e5fc3868e29 from peer 29afadb287c049d9bb777d674b3697ce ({username='slave'} at 127.0.99.134:36071)
I20240501 08:00:05.137344 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet 5c411a11ec484cd5bd063e5fc3868e29 from peer 29afadb287c049d9bb777d674b3697ce at {username='slave'} at 127.0.99.134:36071: session id = 29afadb287c049d9bb777d674b3697ce-5c411a11ec484cd5bd063e5fc3868e29
I20240501 08:00:05.138152 6483 tablet_copy_source_session.cc:205] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.138967 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c411a11ec484cd5bd063e5fc3868e29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.141544 11582 tablet_copy_client.cc:783] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.141712 11582 tablet_copy_client.cc:647] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.142886 11586 tablet_copy_client.cc:1002] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.153647 11602 ts_tablet_manager.cc:902] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 08:00:05.154219 11602 tablet_copy_client.cc:308] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 08:00:05.154702 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet de48f8c2870f4517b5349c30a51fce56 from peer 29afadb287c049d9bb777d674b3697ce ({username='slave'} at 127.0.99.134:51035)
I20240501 08:00:05.154806 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet de48f8c2870f4517b5349c30a51fce56 from peer 29afadb287c049d9bb777d674b3697ce at {username='slave'} at 127.0.99.134:51035: session id = 29afadb287c049d9bb777d674b3697ce-de48f8c2870f4517b5349c30a51fce56
I20240501 08:00:05.155614 6352 tablet_copy_source_session.cc:205] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.156540 11602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de48f8c2870f4517b5349c30a51fce56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.158872 11602 tablet_copy_client.cc:783] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.159096 11602 tablet_copy_client.cc:647] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.160471 11606 tablet_copy_client.cc:1002] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.169412 11617 ts_tablet_manager.cc:902] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a: Initiating tablet copy from peer d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 08:00:05.169987 11617 tablet_copy_client.cc:308] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:42501
I20240501 08:00:05.173851 6614 tablet_copy_service.cc:140] P d2e209265a41425fba98c9d9b0551bd4: Received BeginTabletCopySession request for tablet 2bbceded7f37434ca25b1068bdfc8ca1 from peer 773008dcb81842c3bdf9ce259bb66b8a ({username='slave'} at 127.0.99.137:36345)
I20240501 08:00:05.173961 6614 tablet_copy_service.cc:161] P d2e209265a41425fba98c9d9b0551bd4: Beginning new tablet copy session on tablet 2bbceded7f37434ca25b1068bdfc8ca1 from peer 773008dcb81842c3bdf9ce259bb66b8a at {username='slave'} at 127.0.99.137:36345: session id = 773008dcb81842c3bdf9ce259bb66b8a-2bbceded7f37434ca25b1068bdfc8ca1
I20240501 08:00:05.174926 6614 tablet_copy_source_session.cc:205] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.175731 11628 ts_tablet_manager.cc:902] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 08:00:05.175832 11617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bbceded7f37434ca25b1068bdfc8ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.176398 11628 tablet_copy_client.cc:308] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 08:00:05.176874 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet 8bf587ac1d8141e99fceab28d89e4ea9 from peer 06c63579c7234194b5f6ef457dd5e3f8 ({username='slave'} at 127.0.99.136:47239)
I20240501 08:00:05.176986 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet 8bf587ac1d8141e99fceab28d89e4ea9 from peer 06c63579c7234194b5f6ef457dd5e3f8 at {username='slave'} at 127.0.99.136:47239: session id = 06c63579c7234194b5f6ef457dd5e3f8-8bf587ac1d8141e99fceab28d89e4ea9
I20240501 08:00:05.177840 6352 tablet_copy_source_session.cc:205] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.177984 11617 tablet_copy_client.cc:783] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.178135 11617 tablet_copy_client.cc:647] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.178934 11628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bf587ac1d8141e99fceab28d89e4ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.180708 11628 tablet_copy_client.cc:783] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.180852 11628 tablet_copy_client.cc:647] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.181706 11621 tablet_copy_client.cc:1002] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.181998 11631 tablet_copy_client.cc:1002] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.188377 11638 ts_tablet_manager.cc:902] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8: Initiating tablet copy from peer 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487)
I20240501 08:00:05.188896 11638 tablet_copy_client.cc:308] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:41487
I20240501 08:00:05.189353 6483 tablet_copy_service.cc:140] P 5db2970f4bbd4a3c92d7d913f99716b1: Received BeginTabletCopySession request for tablet 6f24f629734649a3b31c7e293122d7ce from peer 06c63579c7234194b5f6ef457dd5e3f8 ({username='slave'} at 127.0.99.136:51407)
I20240501 08:00:05.189452 6483 tablet_copy_service.cc:161] P 5db2970f4bbd4a3c92d7d913f99716b1: Beginning new tablet copy session on tablet 6f24f629734649a3b31c7e293122d7ce from peer 06c63579c7234194b5f6ef457dd5e3f8 at {username='slave'} at 127.0.99.136:51407: session id = 06c63579c7234194b5f6ef457dd5e3f8-6f24f629734649a3b31c7e293122d7ce
I20240501 08:00:05.190212 6483 tablet_copy_source_session.cc:205] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.191013 11638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f24f629734649a3b31c7e293122d7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.192915 11638 tablet_copy_client.cc:783] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.193055 11638 tablet_copy_client.cc:647] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.194067 11642 tablet_copy_client.cc:1002] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.195039 11643 ts_tablet_manager.cc:902] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 08:00:05.195524 11643 tablet_copy_client.cc:308] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 08:00:05.199218 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet f4c62996a5a64d978850f2f7a31009f4 from peer 773008dcb81842c3bdf9ce259bb66b8a ({username='slave'} at 127.0.99.137:35957)
I20240501 08:00:05.199347 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet f4c62996a5a64d978850f2f7a31009f4 from peer 773008dcb81842c3bdf9ce259bb66b8a at {username='slave'} at 127.0.99.137:35957: session id = 773008dcb81842c3bdf9ce259bb66b8a-f4c62996a5a64d978850f2f7a31009f4
I20240501 08:00:05.200125 6352 tablet_copy_source_session.cc:205] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.200938 11643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c62996a5a64d978850f2f7a31009f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.202694 11643 tablet_copy_client.cc:783] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.202841 11643 tablet_copy_client.cc:647] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.204304 11647 tablet_copy_client.cc:1002] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.222214 11648 ts_tablet_manager.cc:902] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1: Initiating tablet copy from peer d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 08:00:05.222759 11648 tablet_copy_client.cc:308] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:42501
I20240501 08:00:05.223224 6614 tablet_copy_service.cc:140] P d2e209265a41425fba98c9d9b0551bd4: Received BeginTabletCopySession request for tablet 4e052973ae7744e8887d3df0d420f411 from peer ef83010b77b441ffac2e41468a0a1bd1 ({username='slave'} at 127.0.99.135:54555)
I20240501 08:00:05.223322 6614 tablet_copy_service.cc:161] P d2e209265a41425fba98c9d9b0551bd4: Beginning new tablet copy session on tablet 4e052973ae7744e8887d3df0d420f411 from peer ef83010b77b441ffac2e41468a0a1bd1 at {username='slave'} at 127.0.99.135:54555: session id = ef83010b77b441ffac2e41468a0a1bd1-4e052973ae7744e8887d3df0d420f411
I20240501 08:00:05.224064 6614 tablet_copy_source_session.cc:205] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.224891 11648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e052973ae7744e8887d3df0d420f411. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.226655 11648 tablet_copy_client.cc:783] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.226806 11648 tablet_copy_client.cc:647] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.227955 11652 tablet_copy_client.cc:1002] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.247262 11653 ts_tablet_manager.cc:902] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 08:00:05.247841 11653 tablet_copy_client.cc:308] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 08:00:05.248421 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet a4b3aaeeb72e47e08c938b47310dd402 from peer ef83010b77b441ffac2e41468a0a1bd1 ({username='slave'} at 127.0.99.135:48397)
I20240501 08:00:05.248562 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet a4b3aaeeb72e47e08c938b47310dd402 from peer ef83010b77b441ffac2e41468a0a1bd1 at {username='slave'} at 127.0.99.135:48397: session id = ef83010b77b441ffac2e41468a0a1bd1-a4b3aaeeb72e47e08c938b47310dd402
I20240501 08:00:05.249522 6352 tablet_copy_source_session.cc:205] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.250458 11653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b3aaeeb72e47e08c938b47310dd402. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.252674 11653 tablet_copy_client.cc:783] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.252820 11653 tablet_copy_client.cc:647] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.254177 11657 tablet_copy_client.cc:1002] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.264318 11658 ts_tablet_manager.cc:902] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 08:00:05.264861 11658 tablet_copy_client.cc:308] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 08:00:05.265432 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet 41b9f1a284c34a9099cf1014db77f35c from peer 773008dcb81842c3bdf9ce259bb66b8a ({username='slave'} at 127.0.99.137:35957)
I20240501 08:00:05.265544 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet 41b9f1a284c34a9099cf1014db77f35c from peer 773008dcb81842c3bdf9ce259bb66b8a at {username='slave'} at 127.0.99.137:35957: session id = 773008dcb81842c3bdf9ce259bb66b8a-41b9f1a284c34a9099cf1014db77f35c
I20240501 08:00:05.266258 6352 tablet_copy_source_session.cc:205] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.267174 11658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b9f1a284c34a9099cf1014db77f35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.269084 11658 tablet_copy_client.cc:783] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.269222 11658 tablet_copy_client.cc:647] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.270220 11662 tablet_copy_client.cc:1002] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:05.317528 11664 ts_tablet_manager.cc:902] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a: Initiating tablet copy from peer 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 08:00:05.318022 11664 tablet_copy_client.cc:308] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:44189
I20240501 08:00:05.321156 6220 tablet_copy_service.cc:140] P 953b945e3e7e4c71a61aa99c1208ab1d: Received BeginTabletCopySession request for tablet 16752f91ea894a7ba4017170c1dc6ec1 from peer 773008dcb81842c3bdf9ce259bb66b8a ({username='slave'} at 127.0.99.137:33301)
I20240501 08:00:05.321257 6220 tablet_copy_service.cc:161] P 953b945e3e7e4c71a61aa99c1208ab1d: Beginning new tablet copy session on tablet 16752f91ea894a7ba4017170c1dc6ec1 from peer 773008dcb81842c3bdf9ce259bb66b8a at {username='slave'} at 127.0.99.137:33301: session id = 773008dcb81842c3bdf9ce259bb66b8a-16752f91ea894a7ba4017170c1dc6ec1
I20240501 08:00:05.322041 6220 tablet_copy_source_session.cc:205] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:05.322799 11664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16752f91ea894a7ba4017170c1dc6ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:05.324584 11664 tablet_copy_client.cc:783] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:05.324718 11664 tablet_copy_client.cc:647] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:05.325790 11668 tablet_copy_client.cc:1002] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:06.542999 11515 tablet_copy_client.cc:523] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.544862 11515 tablet_bootstrap.cc:492] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap starting.
I20240501 08:00:06.551172 11521 tablet_copy_client.cc:523] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.552696 11521 tablet_bootstrap.cc:492] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap starting.
I20240501 08:00:06.566960 11515 tablet_bootstrap.cc:492] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.567268 11515 tablet_bootstrap.cc:492] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap complete.
I20240501 08:00:06.567442 11515 ts_tablet_manager.cc:1367] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1: Time spent bootstrapping tablet: real 0.023s user 0.019s sys 0.000s
I20240501 08:00:06.567909 11515 raft_consensus.cc:348] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:06.568110 11515 raft_consensus.cc:729] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Initialized, Role: LEARNER
I20240501 08:00:06.568305 11515 consensus_queue.cc:260] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:06.568740 11515 ts_tablet_manager.cc:1398] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.004s
I20240501 08:00:06.569331 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session ef83010b77b441ffac2e41468a0a1bd1-33863ce53ec14aab9040f75e0360e0af received from {username='slave'} at 127.0.99.135:54715
I20240501 08:00:06.569427 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session ef83010b77b441ffac2e41468a0a1bd1-33863ce53ec14aab9040f75e0360e0af on tablet 33863ce53ec14aab9040f75e0360e0af with peer ef83010b77b441ffac2e41468a0a1bd1
I20240501 08:00:06.576222 11521 tablet_bootstrap.cc:492] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.576479 11521 tablet_bootstrap.cc:492] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap complete.
I20240501 08:00:06.576710 11521 ts_tablet_manager.cc:1367] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36: Time spent bootstrapping tablet: real 0.024s user 0.021s sys 0.001s
I20240501 08:00:06.577165 11521 raft_consensus.cc:348] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:06.577283 11521 raft_consensus.cc:729] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 7a80bdedf813425faf7617b4b07c9f36, State: Initialized, Role: LEARNER
I20240501 08:00:06.577358 11521 consensus_queue.cc:260] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:06.577666 11521 ts_tablet_manager.cc:1398] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:06.577666 11526 tablet_copy_client.cc:523] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.578347 6614 tablet_copy_service.cc:342] P d2e209265a41425fba98c9d9b0551bd4: Request end of tablet copy session 7a80bdedf813425faf7617b4b07c9f36-71bcfcbbdfc0491ea936168dd699fc92 received from {username='slave'} at 127.0.99.133:52377
I20240501 08:00:06.578449 6614 tablet_copy_service.cc:433] P d2e209265a41425fba98c9d9b0551bd4: ending tablet copy session 7a80bdedf813425faf7617b4b07c9f36-71bcfcbbdfc0491ea936168dd699fc92 on tablet 71bcfcbbdfc0491ea936168dd699fc92 with peer 7a80bdedf813425faf7617b4b07c9f36
I20240501 08:00:06.579229 11526 tablet_bootstrap.cc:492] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap starting.
I20240501 08:00:06.597201 11531 tablet_copy_client.cc:523] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.599556 11531 tablet_bootstrap.cc:492] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap starting.
I20240501 08:00:06.602032 11526 tablet_bootstrap.cc:492] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.602253 11526 tablet_bootstrap.cc:492] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36: Bootstrap complete.
I20240501 08:00:06.602385 11526 ts_tablet_manager.cc:1367] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36: Time spent bootstrapping tablet: real 0.023s user 0.020s sys 0.001s
I20240501 08:00:06.602897 11526 raft_consensus.cc:348] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:06.603018 11526 raft_consensus.cc:729] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a80bdedf813425faf7617b4b07c9f36, State: Initialized, Role: LEARNER
I20240501 08:00:06.603096 11526 consensus_queue.cc:260] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: true } }
I20240501 08:00:06.603349 11526 ts_tablet_manager.cc:1398] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:06.603989 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session 7a80bdedf813425faf7617b4b07c9f36-98d43e2703b14f9ebf926b5f2e2b2c56 received from {username='slave'} at 127.0.99.133:60211
I20240501 08:00:06.604079 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session 7a80bdedf813425faf7617b4b07c9f36-98d43e2703b14f9ebf926b5f2e2b2c56 on tablet 98d43e2703b14f9ebf926b5f2e2b2c56 with peer 7a80bdedf813425faf7617b4b07c9f36
I20240501 08:00:06.606366 11531 log.cc:824] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:06.621047 11562 tablet_copy_client.cc:523] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.622481 11562 tablet_bootstrap.cc:492] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap starting.
I20240501 08:00:06.632524 11531 tablet_bootstrap.cc:492] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.632905 11531 tablet_bootstrap.cc:492] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap complete.
I20240501 08:00:06.633285 11531 ts_tablet_manager.cc:1367] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent bootstrapping tablet: real 0.034s user 0.032s sys 0.000s
I20240501 08:00:06.636088 11531 raft_consensus.cc:348] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:06.636377 11531 raft_consensus.cc:729] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 773008dcb81842c3bdf9ce259bb66b8a, State: Initialized, Role: LEARNER
I20240501 08:00:06.636835 11531 consensus_queue.cc:260] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:06.637827 11531 ts_tablet_manager.cc:1398] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 08:00:06.638484 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-4e60dad8ddd2402faae35be18662c8c3 received from {username='slave'} at 127.0.99.137:54799
I20240501 08:00:06.638572 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-4e60dad8ddd2402faae35be18662c8c3 on tablet 4e60dad8ddd2402faae35be18662c8c3 with peer 773008dcb81842c3bdf9ce259bb66b8a
I20240501 08:00:06.643177 11582 tablet_copy_client.cc:523] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.645093 11582 tablet_bootstrap.cc:492] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce: Bootstrap starting.
I20240501 08:00:06.648069 11562 tablet_bootstrap.cc:492] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.648373 11562 tablet_bootstrap.cc:492] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap complete.
I20240501 08:00:06.648553 11562 ts_tablet_manager.cc:1367] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8: Time spent bootstrapping tablet: real 0.026s user 0.024s sys 0.000s
I20240501 08:00:06.648963 11562 raft_consensus.cc:348] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:06.649076 11562 raft_consensus.cc:729] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06c63579c7234194b5f6ef457dd5e3f8, State: Initialized, Role: LEARNER
I20240501 08:00:06.649173 11562 consensus_queue.cc:260] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:06.649523 11562 ts_tablet_manager.cc:1398] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:06.650069 6220 tablet_copy_service.cc:342] P 953b945e3e7e4c71a61aa99c1208ab1d: Request end of tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-a97136a7017541eb8e4afb281c97e3cb received from {username='slave'} at 127.0.99.136:46025
I20240501 08:00:06.650148 6220 tablet_copy_service.cc:433] P 953b945e3e7e4c71a61aa99c1208ab1d: ending tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-a97136a7017541eb8e4afb281c97e3cb on tablet a97136a7017541eb8e4afb281c97e3cb with peer 06c63579c7234194b5f6ef457dd5e3f8
I20240501 08:00:06.660969 11602 tablet_copy_client.cc:523] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.662698 11602 tablet_bootstrap.cc:492] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce: Bootstrap starting.
I20240501 08:00:06.664228 7900 raft_consensus.cc:1203] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:06.669699 11582 tablet_bootstrap.cc:492] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.669981 11582 tablet_bootstrap.cc:492] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce: Bootstrap complete.
I20240501 08:00:06.670150 11582 ts_tablet_manager.cc:1367] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce: Time spent bootstrapping tablet: real 0.025s user 0.023s sys 0.000s
I20240501 08:00:06.670545 11582 raft_consensus.cc:348] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:06.670673 11582 raft_consensus.cc:729] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 29afadb287c049d9bb777d674b3697ce, State: Initialized, Role: LEARNER
I20240501 08:00:06.670768 11582 consensus_queue.cc:260] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:06.671126 11582 ts_tablet_manager.cc:1398] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 08:00:06.671732 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session 29afadb287c049d9bb777d674b3697ce-5c411a11ec484cd5bd063e5fc3868e29 received from {username='slave'} at 127.0.99.134:36071
I20240501 08:00:06.671818 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session 29afadb287c049d9bb777d674b3697ce-5c411a11ec484cd5bd063e5fc3868e29 on tablet 5c411a11ec484cd5bd063e5fc3868e29 with peer 29afadb287c049d9bb777d674b3697ce
I20240501 08:00:06.682049 11617 tablet_copy_client.cc:523] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.683665 11617 tablet_bootstrap.cc:492] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap starting.
I20240501 08:00:06.684329 11628 tablet_copy_client.cc:523] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.685868 11628 tablet_bootstrap.cc:492] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap starting.
I20240501 08:00:06.692404 11602 tablet_bootstrap.cc:492] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.692729 11602 tablet_bootstrap.cc:492] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce: Bootstrap complete.
I20240501 08:00:06.692940 11602 ts_tablet_manager.cc:1367] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce: Time spent bootstrapping tablet: real 0.030s user 0.026s sys 0.000s
I20240501 08:00:06.693434 11602 raft_consensus.cc:348] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:06.693570 11602 raft_consensus.cc:729] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 29afadb287c049d9bb777d674b3697ce, State: Initialized, Role: LEARNER
I20240501 08:00:06.693675 11602 consensus_queue.cc:260] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: NON_VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: true } }
I20240501 08:00:06.693971 11602 ts_tablet_manager.cc:1398] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:06.694307 11638 tablet_copy_client.cc:523] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.695859 11638 tablet_bootstrap.cc:492] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap starting.
I20240501 08:00:06.696229 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session 29afadb287c049d9bb777d674b3697ce-de48f8c2870f4517b5349c30a51fce56 received from {username='slave'} at 127.0.99.134:51035
I20240501 08:00:06.696314 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session 29afadb287c049d9bb777d674b3697ce-de48f8c2870f4517b5349c30a51fce56 on tablet de48f8c2870f4517b5349c30a51fce56 with peer 29afadb287c049d9bb777d674b3697ce
I20240501 08:00:06.701751 8162 raft_consensus.cc:1203] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:06.704625 11643 tablet_copy_client.cc:523] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.706454 11643 tablet_bootstrap.cc:492] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap starting.
I20240501 08:00:06.719185 11617 tablet_bootstrap.cc:492] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.719573 11617 tablet_bootstrap.cc:492] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap complete.
I20240501 08:00:06.719884 11617 ts_tablet_manager.cc:1367] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent bootstrapping tablet: real 0.036s user 0.022s sys 0.004s
I20240501 08:00:06.720774 11617 raft_consensus.cc:348] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:06.720948 11617 raft_consensus.cc:729] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 773008dcb81842c3bdf9ce259bb66b8a, State: Initialized, Role: LEARNER
I20240501 08:00:06.721067 11617 consensus_queue.cc:260] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:06.721829 11617 ts_tablet_manager.cc:1398] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:06.722381 6614 tablet_copy_service.cc:342] P d2e209265a41425fba98c9d9b0551bd4: Request end of tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-2bbceded7f37434ca25b1068bdfc8ca1 received from {username='slave'} at 127.0.99.137:36345
I20240501 08:00:06.722469 6614 tablet_copy_service.cc:433] P d2e209265a41425fba98c9d9b0551bd4: ending tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-2bbceded7f37434ca25b1068bdfc8ca1 on tablet 2bbceded7f37434ca25b1068bdfc8ca1 with peer 773008dcb81842c3bdf9ce259bb66b8a
I20240501 08:00:06.727811 11628 tablet_bootstrap.cc:492] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.728267 11648 tablet_copy_client.cc:523] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.729028 11628 tablet_bootstrap.cc:492] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap complete.
I20240501 08:00:06.729225 11628 ts_tablet_manager.cc:1367] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8: Time spent bootstrapping tablet: real 0.043s user 0.025s sys 0.004s
I20240501 08:00:06.729715 11628 raft_consensus.cc:348] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:06.729856 11628 raft_consensus.cc:729] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06c63579c7234194b5f6ef457dd5e3f8, State: Initialized, Role: LEARNER
I20240501 08:00:06.729899 11648 tablet_bootstrap.cc:492] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap starting.
I20240501 08:00:06.729957 11628 consensus_queue.cc:260] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:06.731352 11628 ts_tablet_manager.cc:1398] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:06.731864 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-8bf587ac1d8141e99fceab28d89e4ea9 received from {username='slave'} at 127.0.99.136:47239
I20240501 08:00:06.731988 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-8bf587ac1d8141e99fceab28d89e4ea9 on tablet 8bf587ac1d8141e99fceab28d89e4ea9 with peer 06c63579c7234194b5f6ef457dd5e3f8
I20240501 08:00:06.741699 11643 tablet_bootstrap.cc:492] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.742079 11643 tablet_bootstrap.cc:492] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap complete.
I20240501 08:00:06.742293 11643 ts_tablet_manager.cc:1367] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent bootstrapping tablet: real 0.036s user 0.025s sys 0.004s
I20240501 08:00:06.742821 11643 raft_consensus.cc:348] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:06.742923 11638 tablet_bootstrap.cc:492] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.742960 11643 raft_consensus.cc:729] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 773008dcb81842c3bdf9ce259bb66b8a, State: Initialized, Role: LEARNER
I20240501 08:00:06.743062 11643 consensus_queue.cc:260] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:06.743191 11638 tablet_bootstrap.cc:492] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8: Bootstrap complete.
I20240501 08:00:06.743358 11643 ts_tablet_manager.cc:1398] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:06.743369 11638 ts_tablet_manager.cc:1367] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8: Time spent bootstrapping tablet: real 0.048s user 0.024s sys 0.000s
I20240501 08:00:06.743803 11638 raft_consensus.cc:348] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:06.743953 11638 raft_consensus.cc:729] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06c63579c7234194b5f6ef457dd5e3f8, State: Initialized, Role: LEARNER
I20240501 08:00:06.744053 11638 consensus_queue.cc:260] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: true } }
I20240501 08:00:06.744750 11638 ts_tablet_manager.cc:1398] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8: Time spent starting tablet: real 0.001s user 0.001s sys 0.004s
I20240501 08:00:06.744928 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-f4c62996a5a64d978850f2f7a31009f4 received from {username='slave'} at 127.0.99.137:35957
I20240501 08:00:06.745021 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-f4c62996a5a64d978850f2f7a31009f4 on tablet f4c62996a5a64d978850f2f7a31009f4 with peer 773008dcb81842c3bdf9ce259bb66b8a
I20240501 08:00:06.745311 6483 tablet_copy_service.cc:342] P 5db2970f4bbd4a3c92d7d913f99716b1: Request end of tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-6f24f629734649a3b31c7e293122d7ce received from {username='slave'} at 127.0.99.136:51407
I20240501 08:00:06.745395 6483 tablet_copy_service.cc:433] P 5db2970f4bbd4a3c92d7d913f99716b1: ending tablet copy session 06c63579c7234194b5f6ef457dd5e3f8-6f24f629734649a3b31c7e293122d7ce on tablet 6f24f629734649a3b31c7e293122d7ce with peer 06c63579c7234194b5f6ef457dd5e3f8
I20240501 08:00:06.747519 9189 raft_consensus.cc:1203] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:06.754448 11653 tablet_copy_client.cc:523] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.756147 11653 tablet_bootstrap.cc:492] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap starting.
I20240501 08:00:06.760262 9189 raft_consensus.cc:1203] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:06.770521 11658 tablet_copy_client.cc:523] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.772552 11658 tablet_bootstrap.cc:492] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap starting.
I20240501 08:00:06.770296 11648 tablet_bootstrap.cc:492] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.776432 11648 tablet_bootstrap.cc:492] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap complete.
I20240501 08:00:06.776700 11648 ts_tablet_manager.cc:1367] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1: Time spent bootstrapping tablet: real 0.047s user 0.029s sys 0.000s
I20240501 08:00:06.777328 11648 raft_consensus.cc:348] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:06.777617 11648 raft_consensus.cc:729] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Initialized, Role: LEARNER
I20240501 08:00:06.777778 11648 consensus_queue.cc:260] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:06.778121 11648 ts_tablet_manager.cc:1398] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:06.778784 6614 tablet_copy_service.cc:342] P d2e209265a41425fba98c9d9b0551bd4: Request end of tablet copy session ef83010b77b441ffac2e41468a0a1bd1-4e052973ae7744e8887d3df0d420f411 received from {username='slave'} at 127.0.99.135:54555
I20240501 08:00:06.778867 6614 tablet_copy_service.cc:433] P d2e209265a41425fba98c9d9b0551bd4: ending tablet copy session ef83010b77b441ffac2e41468a0a1bd1-4e052973ae7744e8887d3df0d420f411 on tablet 4e052973ae7744e8887d3df0d420f411 with peer ef83010b77b441ffac2e41468a0a1bd1
I20240501 08:00:06.791553 11653 tablet_bootstrap.cc:492] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.792017 11653 tablet_bootstrap.cc:492] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1: Bootstrap complete.
I20240501 08:00:06.792234 11653 ts_tablet_manager.cc:1367] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1: Time spent bootstrapping tablet: real 0.036s user 0.025s sys 0.004s
I20240501 08:00:06.792726 11653 raft_consensus.cc:348] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:06.792863 11653 raft_consensus.cc:729] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Initialized, Role: LEARNER
I20240501 08:00:06.792963 11653 consensus_queue.cc:260] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: NON_VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: true } }
I20240501 08:00:06.793332 11653 ts_tablet_manager.cc:1398] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:06.794883 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session ef83010b77b441ffac2e41468a0a1bd1-a4b3aaeeb72e47e08c938b47310dd402 received from {username='slave'} at 127.0.99.135:48397
I20240501 08:00:06.794973 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session ef83010b77b441ffac2e41468a0a1bd1-a4b3aaeeb72e47e08c938b47310dd402 on tablet a4b3aaeeb72e47e08c938b47310dd402 with peer ef83010b77b441ffac2e41468a0a1bd1
I20240501 08:00:06.795626 8293 raft_consensus.cc:1203] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:06.807155 8162 raft_consensus.cc:1203] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:06.810204 11658 tablet_bootstrap.cc:492] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.810487 11658 tablet_bootstrap.cc:492] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap complete.
I20240501 08:00:06.810739 11658 ts_tablet_manager.cc:1367] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a: Time spent bootstrapping tablet: real 0.038s user 0.021s sys 0.000s
I20240501 08:00:06.811182 11658 raft_consensus.cc:348] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:06.811293 11658 raft_consensus.cc:729] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 773008dcb81842c3bdf9ce259bb66b8a, State: Initialized, Role: LEARNER
I20240501 08:00:06.811388 11658 consensus_queue.cc:260] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:06.811641 11658 ts_tablet_manager.cc:1398] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:06.812368 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-41b9f1a284c34a9099cf1014db77f35c received from {username='slave'} at 127.0.99.137:35957
I20240501 08:00:06.812450 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-41b9f1a284c34a9099cf1014db77f35c on tablet 41b9f1a284c34a9099cf1014db77f35c with peer 773008dcb81842c3bdf9ce259bb66b8a
I20240501 08:00:06.819098 8293 raft_consensus.cc:1203] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:06.826131 11664 tablet_copy_client.cc:523] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:06.827503 11664 tablet_bootstrap.cc:492] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap starting.
I20240501 08:00:06.847950 11664 tablet_bootstrap.cc:492] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:06.848405 11664 tablet_bootstrap.cc:492] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap complete.
I20240501 08:00:06.848699 11664 ts_tablet_manager.cc:1367] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent bootstrapping tablet: real 0.021s user 0.013s sys 0.008s
I20240501 08:00:06.849268 11664 raft_consensus.cc:348] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:06.849399 11664 raft_consensus.cc:729] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 773008dcb81842c3bdf9ce259bb66b8a, State: Initialized, Role: LEARNER
I20240501 08:00:06.849506 11664 consensus_queue.cc:260] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:06.850085 11664 ts_tablet_manager.cc:1398] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 08:00:06.850208 8031 raft_consensus.cc:1203] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:06.850754 6220 tablet_copy_service.cc:342] P 953b945e3e7e4c71a61aa99c1208ab1d: Request end of tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-16752f91ea894a7ba4017170c1dc6ec1 received from {username='slave'} at 127.0.99.137:33301
I20240501 08:00:06.850845 6220 tablet_copy_service.cc:433] P 953b945e3e7e4c71a61aa99c1208ab1d: ending tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-16752f91ea894a7ba4017170c1dc6ec1 on tablet 16752f91ea894a7ba4017170c1dc6ec1 with peer 773008dcb81842c3bdf9ce259bb66b8a
I20240501 08:00:06.947252 7900 raft_consensus.cc:1203] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.7->[2.8-2.8] Dedup: 2.8->[]
I20240501 08:00:06.999468 9189 raft_consensus.cc:1203] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:07.059423 9189 raft_consensus.cc:1203] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:07.073891 11217 raft_consensus.cc:1050] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 to VOTER
I20240501 08:00:07.074379 11217 consensus_queue.cc:237] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:07.076268 6329 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.076666 7900 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.076727 12129 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.075964 8031 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.077134 12128 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.077558 12128 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.080214 12128 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.080859 8031 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.080907 7900 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.082995 6329 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.083566 6057 catalog_manager.cc:5484] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 7 to 8, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:07.089318 6462 consensus_queue.cc:237] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:07.090679 8031 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.090684 7900 raft_consensus.cc:1261] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.091147 12122 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.091743 11217 consensus_queue.cc:1035] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.094076 12129 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.094511 8030 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.094511 7900 raft_consensus.cc:2932] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.095988 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 98d43e2703b14f9ebf926b5f2e2b2c56 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:07.097635 6051 catalog_manager.cc:5484] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 7a80bdedf813425faf7617b4b07c9f36 reported cstate change: config changed from index 8 to 9, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.100476 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet 98d43e2703b14f9ebf926b5f2e2b2c56 with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:60224
I20240501 08:00:07.100920 12143 tablet_replica.cc:330] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 08:00:07.101336 12143 raft_consensus.cc:2229] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:07.101577 12143 raft_consensus.cc:2258] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:07.102356 12143 ts_tablet_manager.cc:1875] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:07.105206 12143 ts_tablet_manager.cc:1888] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 08:00:07.105307 12143 log.cc:1196] T 98d43e2703b14f9ebf926b5f2e2b2c56 P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/98d43e2703b14f9ebf926b5f2e2b2c56
I20240501 08:00:07.105855 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet 98d43e2703b14f9ebf926b5f2e2b2c56 (table rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]) successfully deleted
I20240501 08:00:07.127638 8293 raft_consensus.cc:1203] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:07.148322 12155 raft_consensus.cc:1050] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4: attempting to promote NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a to VOTER
I20240501 08:00:07.148762 12155 consensus_queue.cc:237] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:07.150209 8293 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.150223 9189 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.150645 12157 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.150784 6198 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.150914 12155 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.151113 12154 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.153780 12157 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.154323 8293 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.155392 9189 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.156323 6198 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.158422 6055 catalog_manager.cc:5484] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:07.162081 6593 consensus_queue.cc:237] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:07.162822 12122 raft_consensus.cc:1050] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 to VOTER
I20240501 08:00:07.163266 12122 consensus_queue.cc:237] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:07.164585 7900 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.164585 8162 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.165000 12129 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.165207 11217 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.165313 9189 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.165411 8293 raft_consensus.cc:1261] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.165971 6329 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.166219 10893 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.166433 12155 consensus_queue.cc:1035] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.166700 12129 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.168649 12155 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.168674 12129 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.169178 8293 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.170351 9189 raft_consensus.cc:2932] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.172019 6051 catalog_manager.cc:5484] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:07.172720 8162 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.172791 7900 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.172951 6059 catalog_manager.cc:5484] T 2bbceded7f37434ca25b1068bdfc8ca1 P 773008dcb81842c3bdf9ce259bb66b8a reported cstate change: config changed from index 8 to 9, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.175307 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2bbceded7f37434ca25b1068bdfc8ca1 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:07.177028 6329 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.178126 6181 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bbceded7f37434ca25b1068bdfc8ca1 with delete_type TABLET_DATA_TOMBSTONED (TS 953b945e3e7e4c71a61aa99c1208ab1d not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:45448
I20240501 08:00:07.178668 12165 tablet_replica.cc:330] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d: stopping tablet replica
I20240501 08:00:07.178864 12165 raft_consensus.cc:2229] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:07.180831 12165 raft_consensus.cc:2258] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:07.181591 12165 ts_tablet_manager.cc:1875] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:07.182516 6462 consensus_queue.cc:237] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:07.183723 7900 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.183763 8162 raft_consensus.cc:1261] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.184064 12165 ts_tablet_manager.cc:1888] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 08:00:07.184139 12165 log.cc:1196] T 2bbceded7f37434ca25b1068bdfc8ca1 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/2bbceded7f37434ca25b1068bdfc8ca1
I20240501 08:00:07.184244 12128 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.184463 12122 consensus_queue.cc:1035] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.184660 6046 catalog_manager.cc:4830] TS 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189): tablet 2bbceded7f37434ca25b1068bdfc8ca1 (table rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]) successfully deleted
I20240501 08:00:07.186137 11319 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.186661 7900 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.186646 8162 raft_consensus.cc:2932] T 33863ce53ec14aab9040f75e0360e0af P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.187845 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 33863ce53ec14aab9040f75e0360e0af with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:07.189481 6056 catalog_manager.cc:5484] T 33863ce53ec14aab9040f75e0360e0af P 7a80bdedf813425faf7617b4b07c9f36 reported cstate change: config changed from index 8 to 9, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.192315 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet 33863ce53ec14aab9040f75e0360e0af with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:60224
I20240501 08:00:07.192584 12143 tablet_replica.cc:330] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 08:00:07.192732 12143 raft_consensus.cc:2229] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:07.192938 12143 raft_consensus.cc:2258] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:07.193624 12143 ts_tablet_manager.cc:1875] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:07.195665 12143 ts_tablet_manager.cc:1888] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 08:00:07.195750 12143 log.cc:1196] T 33863ce53ec14aab9040f75e0360e0af P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/33863ce53ec14aab9040f75e0360e0af
I20240501 08:00:07.196393 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet 33863ce53ec14aab9040f75e0360e0af (table rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]) successfully deleted
I20240501 08:00:07.204173 8030 raft_consensus.cc:1203] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:07.209969 9189 raft_consensus.cc:1203] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:07.213223 12129 raft_consensus.cc:1050] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 to VOTER
I20240501 08:00:07.213742 12129 consensus_queue.cc:237] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:07.215253 7900 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.215305 8293 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.215420 6593 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.215765 12122 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.215977 12129 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.216172 12128 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.218067 12128 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.218504 7900 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.218597 8293 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.220858 6593 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.221643 6056 catalog_manager.cc:5484] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 reported cstate change: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.264576 8162 raft_consensus.cc:1203] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:07.273164 12170 raft_consensus.cc:1050] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d: attempting to promote NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 to VOTER
I20240501 08:00:07.273626 12170 consensus_queue.cc:237] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:07.275290 8030 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.275290 6462 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.275460 8293 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.275656 12172 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.275842 12170 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.276115 12060 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.278286 12060 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.279011 8293 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.279011 6462 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.281515 8030 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.281737 6057 catalog_manager.cc:5484] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8 reported cstate change: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.285586 6198 consensus_queue.cc:237] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:07.286828 8030 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.287240 8293 raft_consensus.cc:1261] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.287463 12172 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.287783 12170 consensus_queue.cc:1035] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.289419 12172 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.289875 8293 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.289932 8030 raft_consensus.cc:2932] T a97136a7017541eb8e4afb281c97e3cb P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.291183 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a97136a7017541eb8e4afb281c97e3cb with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:07.292969 6056 catalog_manager.cc:5484] T a97136a7017541eb8e4afb281c97e3cb P 06c63579c7234194b5f6ef457dd5e3f8 reported cstate change: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.295771 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet a97136a7017541eb8e4afb281c97e3cb with delete_type TABLET_DATA_TOMBSTONED (TS 5db2970f4bbd4a3c92d7d913f99716b1 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:51996
I20240501 08:00:07.296835 12176 tablet_replica.cc:330] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 08:00:07.296983 12176 raft_consensus.cc:2229] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:07.297173 12176 raft_consensus.cc:2258] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:07.298365 12176 ts_tablet_manager.cc:1875] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:07.300386 12176 ts_tablet_manager.cc:1888] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 08:00:07.300460 12176 log.cc:1196] T a97136a7017541eb8e4afb281c97e3cb P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/a97136a7017541eb8e4afb281c97e3cb
I20240501 08:00:07.300984 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet a97136a7017541eb8e4afb281c97e3cb (table rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]) successfully deleted
I20240501 08:00:07.305896 12178 raft_consensus.cc:1050] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 to VOTER
I20240501 08:00:07.306398 12178 consensus_queue.cc:237] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:07.307894 6593 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.307893 7900 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.308022 8162 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.308393 12180 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.308600 12178 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.308810 12177 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.310930 12177 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.311425 8162 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.311472 7900 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.313057 6593 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.314435 6054 catalog_manager.cc:5484] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.320518 12178 raft_consensus.cc:1050] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a to VOTER
I20240501 08:00:07.321045 12178 consensus_queue.cc:237] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:07.322499 9189 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.322604 6593 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.322949 8162 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.322996 11745 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.323575 11745 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.324285 11745 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.326757 12180 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.327251 6593 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.328900 8162 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.330231 6055 catalog_manager.cc:5484] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:07.332119 9189 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.336848 6329 consensus_queue.cc:237] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:07.337276 12178 raft_consensus.cc:1050] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER 29afadb287c049d9bb777d674b3697ce to VOTER
I20240501 08:00:07.337672 12178 consensus_queue.cc:237] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:07.338073 9189 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.338303 8162 raft_consensus.cc:1261] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.338793 11745 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.339058 8162 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.339228 6593 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.339432 11745 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.339844 8030 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.340340 11745 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.340692 12177 consensus_queue.cc:1035] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.340855 12178 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.341513 12180 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.343250 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f4c62996a5a64d978850f2f7a31009f4 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:07.344033 8162 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.344527 6056 catalog_manager.cc:5484] T f4c62996a5a64d978850f2f7a31009f4 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 8 to 9, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:07.344839 9189 raft_consensus.cc:2932] T f4c62996a5a64d978850f2f7a31009f4 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.346918 12178 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.348918 8030 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.350018 8162 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.350370 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet f4c62996a5a64d978850f2f7a31009f4 with delete_type TABLET_DATA_TOMBSTONED (TS d2e209265a41425fba98c9d9b0551bd4 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:60522
I20240501 08:00:07.351233 12212 tablet_replica.cc:330] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4: stopping tablet replica
I20240501 08:00:07.351549 12212 raft_consensus.cc:2229] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:07.351843 12212 raft_consensus.cc:2258] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:07.353152 6054 catalog_manager.cc:5484] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: config changed from index 7 to 8, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.353652 12212 ts_tablet_manager.cc:1875] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:07.355357 6593 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.356369 12212 ts_tablet_manager.cc:1888] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 08:00:07.356436 12212 log.cc:1196] T f4c62996a5a64d978850f2f7a31009f4 P d2e209265a41425fba98c9d9b0551bd4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/f4c62996a5a64d978850f2f7a31009f4
I20240501 08:00:07.356935 6046 catalog_manager.cc:4830] TS d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501): tablet f4c62996a5a64d978850f2f7a31009f4 (table rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]) successfully deleted
I20240501 08:00:07.359123 12154 raft_consensus.cc:1050] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4: attempting to promote NON_VOTER 7a80bdedf813425faf7617b4b07c9f36 to VOTER
I20240501 08:00:07.359545 12154 consensus_queue.cc:237] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:07.361121 7900 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 LEARNER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:07.361183 6198 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:07.361472 12155 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.361613 8162 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:07.361668 12157 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.361999 12157 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.365212 12157 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.365903 8162 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.367033 7900 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.368768 6056 catalog_manager.cc:5484] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: config changed from index 8 to 9, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.369016 6198 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:07.380906 6329 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "de48f8c2870f4517b5349c30a51fce56"
dest_uuid: "4a155eaa444d440d90fde420951458e4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:42062
I20240501 08:00:07.381071 6329 raft_consensus.cc:595] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:07.411617 6462 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6f24f629734649a3b31c7e293122d7ce"
dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:59984
I20240501 08:00:07.411832 6462 raft_consensus.cc:595] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:07.423959 6329 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a4b3aaeeb72e47e08c938b47310dd402"
dest_uuid: "4a155eaa444d440d90fde420951458e4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:42102
I20240501 08:00:07.424427 6329 raft_consensus.cc:595] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:07.455106 6593 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92"
dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55978
I20240501 08:00:07.455269 6593 raft_consensus.cc:595] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 2 LEADER]: Received request to transfer leadership
I20240501 08:00:07.470590 8162 consensus_queue.cc:237] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:07.472671 6329 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240501 08:00:07.472672 6198 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240501 08:00:07.473196 12300 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.473441 12299 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.475279 12299 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:07.475692 6329 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:07.476008 6198 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
W20240501 08:00:07.478826 8095 consensus_peers.cc:485] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 -> Peer 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389): Couldn't send request to peer 773008dcb81842c3bdf9ce259bb66b8a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 96386dd2c62e45859e1f3b30510e8969. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:07.478969 6056 catalog_manager.cc:5484] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 7 to 8, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New cstate: current_term: 2 leader_uuid: "ef83010b77b441ffac2e41468a0a1bd1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:07.487354 6198 consensus_queue.cc:237] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:07.488947 6329 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:07.489003 8293 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 08:00:07.489321 6134 consensus_peers.cc:485] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d -> Peer 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389): Couldn't send request to peer 773008dcb81842c3bdf9ce259bb66b8a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6b552a44c67940819f9ef64c68585eb1. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:07.489343 12170 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:07.489547 12172 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:07.491425 12060 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:07.491997 6329 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:07.493060 8293 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:07.494426 6055 catalog_manager.cc:5484] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:07.525586 12180 raft_consensus.cc:1050] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a to VOTER
I20240501 08:00:07.526073 12180 consensus_queue.cc:237] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:07.527611 8030 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.527597 9189 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.527788 6462 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.528146 11745 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.528384 12177 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.528589 12180 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.530551 12177 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.531445 8030 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.532873 9189 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.534199 6054 catalog_manager.cc:5484] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:07.536351 6462 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.539652 6329 consensus_queue.cc:237] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:07.540844 9189 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.540943 8030 raft_consensus.cc:1261] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.541222 11745 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.541437 12180 consensus_queue.cc:1035] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.541944 6329 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "de48f8c2870f4517b5349c30a51fce56"
dest_uuid: "4a155eaa444d440d90fde420951458e4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:42134
I20240501 08:00:07.542083 6329 raft_consensus.cc:595] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:07.543146 11745 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.543493 9189 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.545424 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 41b9f1a284c34a9099cf1014db77f35c with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:07.545709 6056 catalog_manager.cc:5484] T 41b9f1a284c34a9099cf1014db77f35c P 773008dcb81842c3bdf9ce259bb66b8a reported cstate change: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.546353 8030 raft_consensus.cc:2932] T 41b9f1a284c34a9099cf1014db77f35c P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.548244 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b9f1a284c34a9099cf1014db77f35c with delete_type TABLET_DATA_TOMBSTONED (TS 5db2970f4bbd4a3c92d7d913f99716b1 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:51996
I20240501 08:00:07.548496 12176 tablet_replica.cc:330] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 08:00:07.548663 12176 raft_consensus.cc:2229] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:07.548861 12176 raft_consensus.cc:2258] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:07.549696 12176 ts_tablet_manager.cc:1875] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:07.551784 12176 ts_tablet_manager.cc:1888] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 08:00:07.551854 12176 log.cc:1196] T 41b9f1a284c34a9099cf1014db77f35c P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/41b9f1a284c34a9099cf1014db77f35c
I20240501 08:00:07.552644 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet 41b9f1a284c34a9099cf1014db77f35c (table rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]) successfully deleted
I20240501 08:00:07.558001 6462 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6f24f629734649a3b31c7e293122d7ce"
dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60028
I20240501 08:00:07.558154 6462 raft_consensus.cc:595] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:07.569451 6329 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a4b3aaeeb72e47e08c938b47310dd402"
dest_uuid: "4a155eaa444d440d90fde420951458e4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:42152
I20240501 08:00:07.569605 6329 raft_consensus.cc:595] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:07.598562 12122 raft_consensus.cc:1050] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a to VOTER
I20240501 08:00:07.598860 12177 raft_consensus.cc:1050] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER 06c63579c7234194b5f6ef457dd5e3f8 to VOTER
I20240501 08:00:07.598999 12122 consensus_queue.cc:237] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:07.599251 12177 consensus_queue.cc:237] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:07.600622 8162 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.600899 6593 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.600908 8293 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.601121 8294 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.601193 11745 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.601401 12180 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.601707 9189 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.601795 12180 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.602416 6329 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.603273 11319 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.603554 6592 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92"
dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56004
I20240501 08:00:07.603615 11319 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.603693 6592 raft_consensus.cc:595] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 2 LEADER]: Received request to transfer leadership
I20240501 08:00:07.603909 11319 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.605564 12180 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.605993 6593 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.608026 8293 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.608682 6056 catalog_manager.cc:5484] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.610181 8162 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.615638 6331 consensus_queue.cc:237] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:07.616820 11319 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.617318 8294 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.617563 8162 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.617498 6329 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.618990 9189 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.619488 8294 raft_consensus.cc:1261] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.619489 12177 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.619988 12177 consensus_queue.cc:1035] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.620761 6055 catalog_manager.cc:5484] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:07.622498 11745 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.622946 8162 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.623826 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8bf587ac1d8141e99fceab28d89e4ea9 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:07.625490 6055 catalog_manager.cc:5484] T 8bf587ac1d8141e99fceab28d89e4ea9 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: config changed from index 8 to 9, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.627231 8294 raft_consensus.cc:2932] T 8bf587ac1d8141e99fceab28d89e4ea9 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:07.628369 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bf587ac1d8141e99fceab28d89e4ea9 with delete_type TABLET_DATA_TOMBSTONED (TS d2e209265a41425fba98c9d9b0551bd4 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:60522
I20240501 08:00:07.628592 12212 tablet_replica.cc:330] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4: stopping tablet replica
I20240501 08:00:07.628721 12212 raft_consensus.cc:2229] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:07.628957 12212 raft_consensus.cc:2258] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:07.629854 12212 ts_tablet_manager.cc:1875] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:07.631873 12212 ts_tablet_manager.cc:1888] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 08:00:07.631974 12212 log.cc:1196] T 8bf587ac1d8141e99fceab28d89e4ea9 P d2e209265a41425fba98c9d9b0551bd4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/8bf587ac1d8141e99fceab28d89e4ea9
I20240501 08:00:07.632419 6046 catalog_manager.cc:4830] TS d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501): tablet 8bf587ac1d8141e99fceab28d89e4ea9 (table rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]) successfully deleted
I20240501 08:00:07.645218 12129 raft_consensus.cc:979] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: : Instructing follower 06c63579c7234194b5f6ef457dd5e3f8 to start an election
I20240501 08:00:07.645350 12129 raft_consensus.cc:1067] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Signalling peer 06c63579c7234194b5f6ef457dd5e3f8 to start an election
I20240501 08:00:07.646015 8294 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6f24f629734649a3b31c7e293122d7ce"
dest_uuid: "06c63579c7234194b5f6ef457dd5e3f8"
from {username='slave'} at 127.0.99.131:52035
I20240501 08:00:07.646210 8294 raft_consensus.cc:482] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:07.646307 8294 raft_consensus.cc:3037] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:07.647145 8294 raft_consensus.cc:504] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:07.648221 8294 leader_election.cc:288] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [CANDIDATE]: Term 2 election: Requested vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247)
I20240501 08:00:07.648648 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f24f629734649a3b31c7e293122d7ce" candidate_uuid: "06c63579c7234194b5f6ef457dd5e3f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 08:00:07.648665 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f24f629734649a3b31c7e293122d7ce" candidate_uuid: "06c63579c7234194b5f6ef457dd5e3f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 08:00:07.648789 6462 raft_consensus.cc:3032] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:00:07.648804 6593 raft_consensus.cc:3037] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:07.648835 6462 raft_consensus.cc:729] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Running, Role: LEADER
I20240501 08:00:07.649030 6462 consensus_queue.cc:260] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:07.649223 6462 raft_consensus.cc:3037] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:07.649924 6593 raft_consensus.cc:2454] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06c63579c7234194b5f6ef457dd5e3f8 in term 2.
I20240501 08:00:07.650372 6462 raft_consensus.cc:2454] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06c63579c7234194b5f6ef457dd5e3f8 in term 2.
I20240501 08:00:07.650672 8230 leader_election.cc:302] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06c63579c7234194b5f6ef457dd5e3f8, 5db2970f4bbd4a3c92d7d913f99716b1, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 08:00:07.650943 12000 raft_consensus.cc:2781] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:00:07.651136 12000 raft_consensus.cc:686] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [term 2 LEADER]: Becoming Leader. State: Replica: 06c63579c7234194b5f6ef457dd5e3f8, State: Running, Role: LEADER
I20240501 08:00:07.651456 12000 consensus_queue.cc:237] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:07.653311 7900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f24f629734649a3b31c7e293122d7ce" candidate_uuid: "06c63579c7234194b5f6ef457dd5e3f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "7a80bdedf813425faf7617b4b07c9f36"
I20240501 08:00:07.653466 7900 raft_consensus.cc:3037] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:07.654417 7900 raft_consensus.cc:2454] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06c63579c7234194b5f6ef457dd5e3f8 in term 2.
I20240501 08:00:07.654423 6056 catalog_manager.cc:5484] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 reported cstate change: term changed from 1 to 2, leader changed from 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) to 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136). New cstate: current_term: 2 leader_uuid: "06c63579c7234194b5f6ef457dd5e3f8" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:07.657472 12128 raft_consensus.cc:1050] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1: attempting to promote NON_VOTER 29afadb287c049d9bb777d674b3697ce to VOTER
I20240501 08:00:07.657974 12128 consensus_queue.cc:237] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:07.659590 8030 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce [term 1 LEARNER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.659600 6198 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.660082 12129 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.660142 7900 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.660313 12128 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.660514 11217 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.662258 11217 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.662683 7900 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.662927 6198 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.664259 8030 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.665196 6055 catalog_manager.cc:5484] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 reported cstate change: config changed from index 7 to 8, 29afadb287c049d9bb777d674b3697ce (127.0.99.134) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:07.668061 6462 consensus_queue.cc:237] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:07.669459 7900 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.669478 8030 raft_consensus.cc:1261] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 5db2970f4bbd4a3c92d7d913f99716b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:07.669823 11319 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.670181 12129 consensus_queue.cc:1035] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:07.671478 11217 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.671990 7900 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.673076 8030 raft_consensus.cc:2932] T 5c411a11ec484cd5bd063e5fc3868e29 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.673797 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5c411a11ec484cd5bd063e5fc3868e29 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:07.674336 6057 catalog_manager.cc:5484] T 5c411a11ec484cd5bd063e5fc3868e29 P 7a80bdedf813425faf7617b4b07c9f36 reported cstate change: config changed from index 8 to 9, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:07.676859 6181 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c411a11ec484cd5bd063e5fc3868e29 with delete_type TABLET_DATA_TOMBSTONED (TS 953b945e3e7e4c71a61aa99c1208ab1d not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:45448
I20240501 08:00:07.677153 12165 tablet_replica.cc:330] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d: stopping tablet replica
I20240501 08:00:07.677309 12165 raft_consensus.cc:2229] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:07.677482 12165 raft_consensus.cc:2258] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:07.678181 12165 ts_tablet_manager.cc:1875] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:07.680016 12165 ts_tablet_manager.cc:1888] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 08:00:07.680087 12165 log.cc:1196] T 5c411a11ec484cd5bd063e5fc3868e29 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/5c411a11ec484cd5bd063e5fc3868e29
I20240501 08:00:07.680573 6046 catalog_manager.cc:4830] TS 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189): tablet 5c411a11ec484cd5bd063e5fc3868e29 (table rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]) successfully deleted
I20240501 08:00:07.693502 12170 raft_consensus.cc:1050] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: attempting to promote NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a to VOTER
I20240501 08:00:07.693502 12155 raft_consensus.cc:1050] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: attempting to promote NON_VOTER ef83010b77b441ffac2e41468a0a1bd1 to VOTER
I20240501 08:00:07.693921 12170 consensus_queue.cc:237] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:07.693922 12155 consensus_queue.cc:237] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:07.695338 9189 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.695353 8162 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.695722 12172 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.695811 8162 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 LEARNER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.695819 6329 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.696223 12154 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.696384 8030 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.696382 12172 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.696545 10893 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.697556 10893 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.698005 6462 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:07.698376 12172 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:07.698382 12157 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.699023 8161 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.699420 12172 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.700398 6329 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.701635 8161 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.702199 9189 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.703708 6057 catalog_manager.cc:5484] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:07.704211 6056 catalog_manager.cc:5484] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:07.705323 8030 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:07.705715 6462 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:07.706640 11745 raft_consensus.cc:979] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: : Instructing follower ef83010b77b441ffac2e41468a0a1bd1 to start an election
I20240501 08:00:07.706730 11745 raft_consensus.cc:1067] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Signalling peer ef83010b77b441ffac2e41468a0a1bd1 to start an election
I20240501 08:00:07.707362 8161 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a4b3aaeeb72e47e08c938b47310dd402"
dest_uuid: "ef83010b77b441ffac2e41468a0a1bd1"
from {username='slave'} at 127.0.99.130:48025
I20240501 08:00:07.707494 8161 raft_consensus.cc:482] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:07.707551 8161 raft_consensus.cc:3037] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:07.708639 8161 raft_consensus.cc:504] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:07.709407 8161 leader_election.cc:288] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [CANDIDATE]: Term 2 election: Requested vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247)
I20240501 08:00:07.710888 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b3aaeeb72e47e08c938b47310dd402" candidate_uuid: "ef83010b77b441ffac2e41468a0a1bd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 08:00:07.711022 6593 raft_consensus.cc:3037] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:07.712085 6593 raft_consensus.cc:2454] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef83010b77b441ffac2e41468a0a1bd1 in term 2.
I20240501 08:00:07.712513 6329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b3aaeeb72e47e08c938b47310dd402" candidate_uuid: "ef83010b77b441ffac2e41468a0a1bd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 08:00:07.712642 6329 raft_consensus.cc:3032] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:00:07.712679 6329 raft_consensus.cc:729] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Running, Role: LEADER
I20240501 08:00:07.712816 6329 consensus_queue.cc:260] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:07.713019 6329 raft_consensus.cc:3037] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:07.714058 6329 raft_consensus.cc:2454] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef83010b77b441ffac2e41468a0a1bd1 in term 2.
I20240501 08:00:07.714454 8098 leader_election.cc:302] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4, ef83010b77b441ffac2e41468a0a1bd1; no voters:
I20240501 08:00:07.714671 12305 raft_consensus.cc:2781] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:00:07.714740 12305 raft_consensus.cc:686] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 LEADER]: Becoming Leader. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Running, Role: LEADER
I20240501 08:00:07.714880 12305 consensus_queue.cc:237] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:07.714970 7900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b3aaeeb72e47e08c938b47310dd402" candidate_uuid: "ef83010b77b441ffac2e41468a0a1bd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "7a80bdedf813425faf7617b4b07c9f36"
I20240501 08:00:07.715096 7900 raft_consensus.cc:3037] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:07.716125 7900 raft_consensus.cc:2454] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef83010b77b441ffac2e41468a0a1bd1 in term 2.
I20240501 08:00:07.717181 6051 catalog_manager.cc:5484] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: term changed from 1 to 2, leader changed from 4a155eaa444d440d90fde420951458e4 (127.0.99.130) to ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135). New cstate: current_term: 2 leader_uuid: "ef83010b77b441ffac2e41468a0a1bd1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:07.798761 12157 raft_consensus.cc:979] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4: : Instructing follower ef83010b77b441ffac2e41468a0a1bd1 to start an election
I20240501 08:00:07.798888 12157 raft_consensus.cc:1067] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 2 LEADER]: Signalling peer ef83010b77b441ffac2e41468a0a1bd1 to start an election
I20240501 08:00:07.799376 8161 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92"
dest_uuid: "ef83010b77b441ffac2e41468a0a1bd1"
from {username='slave'} at 127.0.99.132:53569
I20240501 08:00:07.799510 8161 raft_consensus.cc:482] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:07.799571 8161 raft_consensus.cc:3037] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:00:07.800438 8161 raft_consensus.cc:504] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:07.800938 8161 leader_election.cc:288] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [CANDIDATE]: Term 3 election: Requested vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247)
I20240501 08:00:07.801301 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92" candidate_uuid: "ef83010b77b441ffac2e41468a0a1bd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 08:00:07.801347 7900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92" candidate_uuid: "ef83010b77b441ffac2e41468a0a1bd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "7a80bdedf813425faf7617b4b07c9f36"
I20240501 08:00:07.801416 6593 raft_consensus.cc:3032] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 2 LEADER]: Stepping down as leader of term 2
I20240501 08:00:07.801455 6593 raft_consensus.cc:729] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Running, Role: LEADER
I20240501 08:00:07.801476 7900 raft_consensus.cc:3037] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:00:07.801532 6198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71bcfcbbdfc0491ea936168dd699fc92" candidate_uuid: "ef83010b77b441ffac2e41468a0a1bd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 08:00:07.801596 6593 consensus_queue.cc:260] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:07.801658 6198 raft_consensus.cc:3037] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:00:07.801842 6593 raft_consensus.cc:3037] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Advancing to term 3
I20240501 08:00:07.802464 7900 raft_consensus.cc:2454] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef83010b77b441ffac2e41468a0a1bd1 in term 3.
I20240501 08:00:07.802536 6198 raft_consensus.cc:2454] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef83010b77b441ffac2e41468a0a1bd1 in term 3.
I20240501 08:00:07.802865 8098 leader_election.cc:302] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 7a80bdedf813425faf7617b4b07c9f36, 953b945e3e7e4c71a61aa99c1208ab1d, ef83010b77b441ffac2e41468a0a1bd1; no voters:
I20240501 08:00:07.802989 6593 raft_consensus.cc:2454] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef83010b77b441ffac2e41468a0a1bd1 in term 3.
I20240501 08:00:07.803061 12305 raft_consensus.cc:2781] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 3 FOLLOWER]: Leader election won for term 3
I20240501 08:00:07.803136 12305 raft_consensus.cc:686] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 3 LEADER]: Becoming Leader. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Running, Role: LEADER
I20240501 08:00:07.803285 12305 consensus_queue.cc:237] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:07.805186 6051 catalog_manager.cc:5484] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: term changed from 2 to 3, leader changed from d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) to ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135). New cstate: current_term: 3 leader_uuid: "ef83010b77b441ffac2e41468a0a1bd1" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:07.807330 12177 raft_consensus.cc:979] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: : Instructing follower 29afadb287c049d9bb777d674b3697ce to start an election
I20240501 08:00:07.807431 12177 raft_consensus.cc:1067] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Signalling peer 29afadb287c049d9bb777d674b3697ce to start an election
I20240501 08:00:07.807868 8030 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "de48f8c2870f4517b5349c30a51fce56"
dest_uuid: "29afadb287c049d9bb777d674b3697ce"
from {username='slave'} at 127.0.99.130:57381
I20240501 08:00:07.808092 8030 raft_consensus.cc:482] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:07.808190 8030 raft_consensus.cc:3037] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:07.809010 8030 raft_consensus.cc:504] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:07.810078 8030 leader_election.cc:288] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [CANDIDATE]: Term 2 election: Requested vote from peers 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937)
I20240501 08:00:07.810500 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de48f8c2870f4517b5349c30a51fce56" candidate_uuid: "29afadb287c049d9bb777d674b3697ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 08:00:07.810634 6593 raft_consensus.cc:3037] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:07.811120 6329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de48f8c2870f4517b5349c30a51fce56" candidate_uuid: "29afadb287c049d9bb777d674b3697ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 08:00:07.811244 6329 raft_consensus.cc:3032] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:00:07.811290 6329 raft_consensus.cc:729] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4a155eaa444d440d90fde420951458e4, State: Running, Role: LEADER
I20240501 08:00:07.811450 6329 consensus_queue.cc:260] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:07.811622 6329 raft_consensus.cc:3037] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:07.811667 6593 raft_consensus.cc:2454] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29afadb287c049d9bb777d674b3697ce in term 2.
I20240501 08:00:07.812459 6329 raft_consensus.cc:2454] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29afadb287c049d9bb777d674b3697ce in term 2.
I20240501 08:00:07.812753 7967 leader_election.cc:302] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 29afadb287c049d9bb777d674b3697ce, 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 08:00:07.813066 12012 raft_consensus.cc:2781] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:00:07.813290 12012 raft_consensus.cc:686] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [term 2 LEADER]: Becoming Leader. State: Replica: 29afadb287c049d9bb777d674b3697ce, State: Running, Role: LEADER
I20240501 08:00:07.813584 12012 consensus_queue.cc:237] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:07.814180 8161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de48f8c2870f4517b5349c30a51fce56" candidate_uuid: "29afadb287c049d9bb777d674b3697ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "ef83010b77b441ffac2e41468a0a1bd1"
I20240501 08:00:07.814311 8161 raft_consensus.cc:3037] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:07.816162 6051 catalog_manager.cc:5484] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce reported cstate change: term changed from 1 to 2, leader changed from 4a155eaa444d440d90fde420951458e4 (127.0.99.130) to 29afadb287c049d9bb777d674b3697ce (127.0.99.134). New cstate: current_term: 2 leader_uuid: "29afadb287c049d9bb777d674b3697ce" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:07.828972 8161 raft_consensus.cc:2454] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29afadb287c049d9bb777d674b3697ce in term 2.
I20240501 08:00:07.844959 6198 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "16752f91ea894a7ba4017170c1dc6ec1"
dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37512
I20240501 08:00:07.845099 6198 raft_consensus.cc:595] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:07.855350 6593 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4e052973ae7744e8887d3df0d420f411"
dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56020
I20240501 08:00:07.855502 6593 raft_consensus.cc:595] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:07.902312 6462 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4e60dad8ddd2402faae35be18662c8c3"
dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60068
I20240501 08:00:07.902546 6462 raft_consensus.cc:595] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:07.911401 6198 consensus_queue.cc:237] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:07.912917 8030 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:07.912986 6462 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:07.913349 11746 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240501 08:00:07.913563 6134 consensus_peers.cc:485] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d -> Peer 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389): Couldn't send request to peer 773008dcb81842c3bdf9ce259bb66b8a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d929e6e09be943b68061ad4b1bb8e9f1. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:07.913705 11746 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:07.915199 11746 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:07.915745 6462 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:07.916043 8030 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:07.918236 6051 catalog_manager.cc:5484] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:07.939271 6198 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "16752f91ea894a7ba4017170c1dc6ec1"
dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37548
I20240501 08:00:07.939420 6198 raft_consensus.cc:595] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:07.949834 6593 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4e052973ae7744e8887d3df0d420f411"
dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56044
I20240501 08:00:07.949980 6593 raft_consensus.cc:595] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:07.990514 6462 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4e60dad8ddd2402faae35be18662c8c3"
dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60094
I20240501 08:00:07.990662 6462 raft_consensus.cc:595] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:07.994603 12574 ts_tablet_manager.cc:902] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a: Initiating tablet copy from peer ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937)
I20240501 08:00:07.995194 12574 tablet_copy_client.cc:308] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:34937
I20240501 08:00:07.998942 8182 tablet_copy_service.cc:140] P ef83010b77b441ffac2e41468a0a1bd1: Received BeginTabletCopySession request for tablet 96386dd2c62e45859e1f3b30510e8969 from peer 773008dcb81842c3bdf9ce259bb66b8a ({username='slave'} at 127.0.99.137:54955)
I20240501 08:00:07.999085 8182 tablet_copy_service.cc:161] P ef83010b77b441ffac2e41468a0a1bd1: Beginning new tablet copy session on tablet 96386dd2c62e45859e1f3b30510e8969 from peer 773008dcb81842c3bdf9ce259bb66b8a at {username='slave'} at 127.0.99.137:54955: session id = 773008dcb81842c3bdf9ce259bb66b8a-96386dd2c62e45859e1f3b30510e8969
I20240501 08:00:08.000284 8182 tablet_copy_source_session.cc:205] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:08.001405 12574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96386dd2c62e45859e1f3b30510e8969. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:08.003233 12574 tablet_copy_client.cc:783] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:08.003374 12574 tablet_copy_client.cc:647] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:08.004724 12578 tablet_copy_client.cc:1002] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:08.049463 12122 raft_consensus.cc:979] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: : Instructing follower 06c63579c7234194b5f6ef457dd5e3f8 to start an election
I20240501 08:00:08.049604 12129 raft_consensus.cc:1067] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Signalling peer 06c63579c7234194b5f6ef457dd5e3f8 to start an election
I20240501 08:00:08.050118 8294 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4e60dad8ddd2402faae35be18662c8c3"
dest_uuid: "06c63579c7234194b5f6ef457dd5e3f8"
from {username='slave'} at 127.0.99.131:52035
I20240501 08:00:08.050268 8294 raft_consensus.cc:482] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:08.050330 8294 raft_consensus.cc:3037] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:08.051137 8294 raft_consensus.cc:504] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:08.051715 8294 leader_election.cc:288] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [CANDIDATE]: Term 2 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389)
I20240501 08:00:08.052069 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e60dad8ddd2402faae35be18662c8c3" candidate_uuid: "06c63579c7234194b5f6ef457dd5e3f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 08:00:08.052193 6462 raft_consensus.cc:3032] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:00:08.052160 6329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e60dad8ddd2402faae35be18662c8c3" candidate_uuid: "06c63579c7234194b5f6ef457dd5e3f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 08:00:08.052237 6462 raft_consensus.cc:729] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5db2970f4bbd4a3c92d7d913f99716b1, State: Running, Role: LEADER
I20240501 08:00:08.052291 6329 raft_consensus.cc:3037] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:08.052388 6462 consensus_queue.cc:260] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:08.052579 6462 raft_consensus.cc:3037] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:08.053409 6329 raft_consensus.cc:2454] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06c63579c7234194b5f6ef457dd5e3f8 in term 2.
I20240501 08:00:08.053460 6462 raft_consensus.cc:2454] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06c63579c7234194b5f6ef457dd5e3f8 in term 2.
I20240501 08:00:08.053812 8230 leader_election.cc:302] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06c63579c7234194b5f6ef457dd5e3f8, 4a155eaa444d440d90fde420951458e4, 5db2970f4bbd4a3c92d7d913f99716b1; no voters:
I20240501 08:00:08.054039 12000 raft_consensus.cc:2781] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:00:08.054212 12000 raft_consensus.cc:686] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 2 LEADER]: Becoming Leader. State: Replica: 06c63579c7234194b5f6ef457dd5e3f8, State: Running, Role: LEADER
I20240501 08:00:08.054380 12000 consensus_queue.cc:237] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:08.055137 12581 ts_tablet_manager.cc:902] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a: Initiating tablet copy from peer 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 08:00:08.056604 6059 catalog_manager.cc:5484] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 reported cstate change: term changed from 1 to 2, leader changed from 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) to 06c63579c7234194b5f6ef457dd5e3f8 (127.0.99.136). New cstate: current_term: 2 leader_uuid: "06c63579c7234194b5f6ef457dd5e3f8" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:08.057469 12581 tablet_copy_client.cc:308] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:44189
I20240501 08:00:08.058081 6220 tablet_copy_service.cc:140] P 953b945e3e7e4c71a61aa99c1208ab1d: Received BeginTabletCopySession request for tablet 6b552a44c67940819f9ef64c68585eb1 from peer 773008dcb81842c3bdf9ce259bb66b8a ({username='slave'} at 127.0.99.137:33301)
I20240501 08:00:08.058200 6220 tablet_copy_service.cc:161] P 953b945e3e7e4c71a61aa99c1208ab1d: Beginning new tablet copy session on tablet 6b552a44c67940819f9ef64c68585eb1 from peer 773008dcb81842c3bdf9ce259bb66b8a at {username='slave'} at 127.0.99.137:33301: session id = 773008dcb81842c3bdf9ce259bb66b8a-6b552a44c67940819f9ef64c68585eb1
I20240501 08:00:08.058212 9189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e60dad8ddd2402faae35be18662c8c3" candidate_uuid: "06c63579c7234194b5f6ef457dd5e3f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "773008dcb81842c3bdf9ce259bb66b8a"
I20240501 08:00:08.058415 9189 raft_consensus.cc:3037] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:08.059154 6220 tablet_copy_source_session.cc:205] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:08.059628 9189 raft_consensus.cc:2454] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06c63579c7234194b5f6ef457dd5e3f8 in term 2.
I20240501 08:00:08.060114 12581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b552a44c67940819f9ef64c68585eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:08.061718 12581 tablet_copy_client.cc:783] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:08.061856 12581 tablet_copy_client.cc:647] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:08.063052 12584 tablet_copy_client.cc:1002] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:08.087122 12170 raft_consensus.cc:979] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: : Instructing follower ef83010b77b441ffac2e41468a0a1bd1 to start an election
I20240501 08:00:08.087239 12170 raft_consensus.cc:1067] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Signalling peer ef83010b77b441ffac2e41468a0a1bd1 to start an election
I20240501 08:00:08.087649 8161 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "16752f91ea894a7ba4017170c1dc6ec1"
dest_uuid: "ef83010b77b441ffac2e41468a0a1bd1"
from {username='slave'} at 127.0.99.129:58541
I20240501 08:00:08.087796 8161 raft_consensus.cc:482] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:08.087863 8161 raft_consensus.cc:3037] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:08.088665 8161 raft_consensus.cc:504] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:08.089113 8161 leader_election.cc:288] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [CANDIDATE]: Term 2 election: Requested vote from peers 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487), 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189), 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389)
I20240501 08:00:08.089437 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16752f91ea894a7ba4017170c1dc6ec1" candidate_uuid: "ef83010b77b441ffac2e41468a0a1bd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "5db2970f4bbd4a3c92d7d913f99716b1"
I20240501 08:00:08.089570 6462 raft_consensus.cc:3037] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:08.089533 6198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16752f91ea894a7ba4017170c1dc6ec1" candidate_uuid: "ef83010b77b441ffac2e41468a0a1bd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "953b945e3e7e4c71a61aa99c1208ab1d"
I20240501 08:00:08.089656 6198 raft_consensus.cc:3032] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:00:08.089702 6198 raft_consensus.cc:729] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 953b945e3e7e4c71a61aa99c1208ab1d, State: Running, Role: LEADER
I20240501 08:00:08.089720 9189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16752f91ea894a7ba4017170c1dc6ec1" candidate_uuid: "ef83010b77b441ffac2e41468a0a1bd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "773008dcb81842c3bdf9ce259bb66b8a"
I20240501 08:00:08.089844 9189 raft_consensus.cc:3037] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:08.089862 6198 consensus_queue.cc:260] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:08.090054 6198 raft_consensus.cc:3037] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:08.090595 6462 raft_consensus.cc:2454] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef83010b77b441ffac2e41468a0a1bd1 in term 2.
I20240501 08:00:08.090766 9189 raft_consensus.cc:2454] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef83010b77b441ffac2e41468a0a1bd1 in term 2.
I20240501 08:00:08.090911 6198 raft_consensus.cc:2454] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef83010b77b441ffac2e41468a0a1bd1 in term 2.
I20240501 08:00:08.091091 8095 leader_election.cc:302] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5db2970f4bbd4a3c92d7d913f99716b1, 773008dcb81842c3bdf9ce259bb66b8a, ef83010b77b441ffac2e41468a0a1bd1; no voters:
I20240501 08:00:08.091356 12305 raft_consensus.cc:2781] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:00:08.091475 12305 raft_consensus.cc:686] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 LEADER]: Becoming Leader. State: Replica: ef83010b77b441ffac2e41468a0a1bd1, State: Running, Role: LEADER
I20240501 08:00:08.091614 12305 consensus_queue.cc:237] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:08.093441 6059 catalog_manager.cc:5484] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: term changed from 1 to 2, leader changed from 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) to ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135). New cstate: current_term: 2 leader_uuid: "ef83010b77b441ffac2e41468a0a1bd1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:08.100507 7900 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 2 FOLLOWER]: Refusing update from remote peer 06c63579c7234194b5f6ef457dd5e3f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.101006 12000 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.128727 6462 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 2 FOLLOWER]: Refusing update from remote peer 06c63579c7234194b5f6ef457dd5e3f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.129223 12000 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.133584 6593 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Refusing update from remote peer 06c63579c7234194b5f6ef457dd5e3f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.134021 12000 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.157459 7900 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.157927 12305 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.162611 12154 raft_consensus.cc:979] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: : Instructing follower 29afadb287c049d9bb777d674b3697ce to start an election
I20240501 08:00:08.162734 12154 raft_consensus.cc:1067] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Signalling peer 29afadb287c049d9bb777d674b3697ce to start an election
I20240501 08:00:08.163156 8030 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4e052973ae7744e8887d3df0d420f411"
dest_uuid: "29afadb287c049d9bb777d674b3697ce"
from {username='slave'} at 127.0.99.132:59971
I20240501 08:00:08.163308 8030 raft_consensus.cc:482] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:08.163373 8030 raft_consensus.cc:3037] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:08.164235 8030 raft_consensus.cc:504] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:08.164706 8030 leader_election.cc:288] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [CANDIDATE]: Term 2 election: Requested vote from peers d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501), 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285), ef83010b77b441ffac2e41468a0a1bd1 (127.0.99.135:34937)
I20240501 08:00:08.165037 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e052973ae7744e8887d3df0d420f411" candidate_uuid: "29afadb287c049d9bb777d674b3697ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "d2e209265a41425fba98c9d9b0551bd4"
I20240501 08:00:08.165120 6329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e052973ae7744e8887d3df0d420f411" candidate_uuid: "29afadb287c049d9bb777d674b3697ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "4a155eaa444d440d90fde420951458e4"
I20240501 08:00:08.165159 6593 raft_consensus.cc:3032] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:00:08.165201 6593 raft_consensus.cc:729] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d2e209265a41425fba98c9d9b0551bd4, State: Running, Role: LEADER
I20240501 08:00:08.165235 6329 raft_consensus.cc:3037] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:08.165359 6593 consensus_queue.cc:260] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:08.165544 6593 raft_consensus.cc:3037] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:08.166134 6329 raft_consensus.cc:2454] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29afadb287c049d9bb777d674b3697ce in term 2.
I20240501 08:00:08.166280 6593 raft_consensus.cc:2454] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29afadb287c049d9bb777d674b3697ce in term 2.
I20240501 08:00:08.166383 8161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e052973ae7744e8887d3df0d420f411" candidate_uuid: "29afadb287c049d9bb777d674b3697ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "ef83010b77b441ffac2e41468a0a1bd1"
I20240501 08:00:08.166501 8161 raft_consensus.cc:3037] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:08.166491 7967 leader_election.cc:302] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 29afadb287c049d9bb777d674b3697ce, 4a155eaa444d440d90fde420951458e4, d2e209265a41425fba98c9d9b0551bd4; no voters:
I20240501 08:00:08.166644 12012 raft_consensus.cc:2781] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:00:08.166711 12012 raft_consensus.cc:686] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 2 LEADER]: Becoming Leader. State: Replica: 29afadb287c049d9bb777d674b3697ce, State: Running, Role: LEADER
I20240501 08:00:08.166918 12012 consensus_queue.cc:237] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:08.167290 8161 raft_consensus.cc:2454] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29afadb287c049d9bb777d674b3697ce in term 2.
I20240501 08:00:08.168721 6054 catalog_manager.cc:5484] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce reported cstate change: term changed from 1 to 2, leader changed from d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) to 29afadb287c049d9bb777d674b3697ce (127.0.99.134). New cstate: current_term: 2 leader_uuid: "29afadb287c049d9bb777d674b3697ce" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:08.253156 7900 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36 [term 3 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240501 08:00:08.253558 12305 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.259617 6593 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.260095 12305 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.262511 6329 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.263002 12642 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.265705 8161 consensus_queue.cc:237] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:08.267263 6593 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:08.267604 7900 raft_consensus.cc:1261] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:08.267897 12642 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.268057 12300 consensus_queue.cc:1035] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.269546 12300 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:08.269929 6593 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:08.270205 7900 raft_consensus.cc:2932] T a4b3aaeeb72e47e08c938b47310dd402 P 7a80bdedf813425faf7617b4b07c9f36 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:08.271103 6044 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a4b3aaeeb72e47e08c938b47310dd402 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:08.272598 6051 catalog_manager.cc:5484] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: config changed from index 8 to 10, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 2 leader_uuid: "ef83010b77b441ffac2e41468a0a1bd1" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:08.273864 6593 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 3 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240501 08:00:08.275022 12305 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.275869 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b3aaeeb72e47e08c938b47310dd402 with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:60224
I20240501 08:00:08.276293 12644 tablet_replica.cc:330] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 08:00:08.276650 12644 raft_consensus.cc:2229] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:08.276861 12644 raft_consensus.cc:2258] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:08.277601 12644 ts_tablet_manager.cc:1875] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:08.278358 6198 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 3 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240501 08:00:08.278863 12305 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.280699 12644 ts_tablet_manager.cc:1888] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 08:00:08.280772 12644 log.cc:1196] T a4b3aaeeb72e47e08c938b47310dd402 P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/a4b3aaeeb72e47e08c938b47310dd402
I20240501 08:00:08.281961 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet a4b3aaeeb72e47e08c938b47310dd402 (table rebalance_test_table_5 [id=85f5373daa8f4159afdc56a6399177fc]) successfully deleted
I20240501 08:00:08.311784 12647 ts_tablet_manager.cc:902] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a: Initiating tablet copy from peer 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189)
I20240501 08:00:08.312342 12647 tablet_copy_client.cc:308] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:44189
I20240501 08:00:08.312798 6220 tablet_copy_service.cc:140] P 953b945e3e7e4c71a61aa99c1208ab1d: Received BeginTabletCopySession request for tablet d929e6e09be943b68061ad4b1bb8e9f1 from peer 773008dcb81842c3bdf9ce259bb66b8a ({username='slave'} at 127.0.99.137:33301)
I20240501 08:00:08.312897 6220 tablet_copy_service.cc:161] P 953b945e3e7e4c71a61aa99c1208ab1d: Beginning new tablet copy session on tablet d929e6e09be943b68061ad4b1bb8e9f1 from peer 773008dcb81842c3bdf9ce259bb66b8a at {username='slave'} at 127.0.99.137:33301: session id = 773008dcb81842c3bdf9ce259bb66b8a-d929e6e09be943b68061ad4b1bb8e9f1
I20240501 08:00:08.313598 6220 tablet_copy_source_session.cc:205] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:08.314460 12647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d929e6e09be943b68061ad4b1bb8e9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:08.316208 12647 tablet_copy_client.cc:783] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:08.316360 12647 tablet_copy_client.cc:647] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:08.317370 12651 tablet_copy_client.cc:1002] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:08.327877 6329 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer 29afadb287c049d9bb777d674b3697ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.328446 12012 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.414149 6593 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Refusing update from remote peer 29afadb287c049d9bb777d674b3697ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.414568 12012 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.417332 8161 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Refusing update from remote peer 29afadb287c049d9bb777d674b3697ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.417989 12012 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.471119 6462 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.471550 12642 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.508724 9189 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.509105 12642 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.511808 6198 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.512468 12645 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.514010 8161 consensus_queue.cc:237] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:08.515313 6462 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:08.515666 9188 raft_consensus.cc:1261] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:08.515812 12645 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.516219 12645 consensus_queue.cc:1035] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.518638 12645 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:08.519467 9188 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:08.521265 6044 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 16752f91ea894a7ba4017170c1dc6ec1 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:08.521224 6462 raft_consensus.cc:2932] T 16752f91ea894a7ba4017170c1dc6ec1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:08.521492 6051 catalog_manager.cc:5484] T 16752f91ea894a7ba4017170c1dc6ec1 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: config changed from index 8 to 10, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "ef83010b77b441ffac2e41468a0a1bd1" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:08.525864 6181 tablet_service.cc:1514] Processing DeleteTablet for tablet 16752f91ea894a7ba4017170c1dc6ec1 with delete_type TABLET_DATA_TOMBSTONED (TS 953b945e3e7e4c71a61aa99c1208ab1d not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:45448
I20240501 08:00:08.526168 12720 tablet_replica.cc:330] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: stopping tablet replica
I20240501 08:00:08.526322 12720 raft_consensus.cc:2229] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:08.526446 12720 raft_consensus.cc:2258] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:08.527038 12720 ts_tablet_manager.cc:1875] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:08.529273 12720 ts_tablet_manager.cc:1888] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 08:00:08.529340 12720 log.cc:1196] T 16752f91ea894a7ba4017170c1dc6ec1 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/16752f91ea894a7ba4017170c1dc6ec1
I20240501 08:00:08.529943 6046 catalog_manager.cc:4830] TS 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189): tablet 16752f91ea894a7ba4017170c1dc6ec1 (table rebalance_test_table_4 [id=95137d2c23464e768d3ffbd4956111dd]) successfully deleted
I20240501 08:00:08.555028 6329 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer 29afadb287c049d9bb777d674b3697ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.555473 12655 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.579596 6329 consensus_queue.cc:237] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:08.581029 8294 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:08.581358 6593 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:08.581429 12763 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:08.581717 12177 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:08.583560 11745 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:08.584025 6593 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:08.584064 8294 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
W20240501 08:00:08.586510 6265 consensus_peers.cc:485] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 -> Peer 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389): Couldn't send request to peer 773008dcb81842c3bdf9ce259bb66b8a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 88fa1c4aaf754e32bef123157b3bc62c. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:08.586622 6329 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer 06c63579c7234194b5f6ef457dd5e3f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.587097 6051 catalog_manager.cc:5484] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:08.588140 12587 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.589727 9188 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 FOLLOWER]: Refusing update from remote peer 06c63579c7234194b5f6ef457dd5e3f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.591059 12587 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.594190 6462 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 2 FOLLOWER]: Refusing update from remote peer 06c63579c7234194b5f6ef457dd5e3f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.594693 12408 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.603701 8294 consensus_queue.cc:237] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:08.605034 9188 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 FOLLOWER]: Refusing update from remote peer 06c63579c7234194b5f6ef457dd5e3f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:08.605126 6329 raft_consensus.cc:1261] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer 06c63579c7234194b5f6ef457dd5e3f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:08.605496 12587 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.605634 12000 consensus_queue.cc:1035] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.607415 12000 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:08.607995 6329 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:08.608006 9188 raft_consensus.cc:2932] T 4e60dad8ddd2402faae35be18662c8c3 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:08.610013 6054 catalog_manager.cc:5484] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 reported cstate change: config changed from index 8 to 10, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 2 leader_uuid: "06c63579c7234194b5f6ef457dd5e3f8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:08.610764 6043 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4e60dad8ddd2402faae35be18662c8c3 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:08.613303 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e60dad8ddd2402faae35be18662c8c3 with delete_type TABLET_DATA_TOMBSTONED (TS 5db2970f4bbd4a3c92d7d913f99716b1 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:51996
I20240501 08:00:08.613723 12777 tablet_replica.cc:330] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 08:00:08.613896 12777 raft_consensus.cc:2229] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:08.614001 12777 raft_consensus.cc:2258] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:08.614578 12777 ts_tablet_manager.cc:1875] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:08.616825 12777 ts_tablet_manager.cc:1888] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 08:00:08.616909 12777 log.cc:1196] T 4e60dad8ddd2402faae35be18662c8c3 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/4e60dad8ddd2402faae35be18662c8c3
I20240501 08:00:08.617379 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet 4e60dad8ddd2402faae35be18662c8c3 (table rebalance_test_table_7 [id=14a64aba080c4933a1dad88f9c501f32]) successfully deleted
I20240501 08:00:08.644191 8294 consensus_queue.cc:237] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } }
I20240501 08:00:08.645360 6593 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Refusing update from remote peer 06c63579c7234194b5f6ef457dd5e3f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:08.645449 7900 raft_consensus.cc:1261] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 2 FOLLOWER]: Refusing update from remote peer 06c63579c7234194b5f6ef457dd5e3f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:08.645793 12408 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.646138 12408 consensus_queue.cc:1035] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.647418 12000 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 06c63579c7234194b5f6ef457dd5e3f8 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:08.647979 7900 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:08.648980 6043 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6f24f629734649a3b31c7e293122d7ce with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:08.649909 6593 raft_consensus.cc:2932] T 6f24f629734649a3b31c7e293122d7ce P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:08.650398 6051 catalog_manager.cc:5484] T 6f24f629734649a3b31c7e293122d7ce P 7a80bdedf813425faf7617b4b07c9f36 reported cstate change: config changed from index 8 to 10, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 2 leader_uuid: "06c63579c7234194b5f6ef457dd5e3f8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } }
I20240501 08:00:08.653146 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f24f629734649a3b31c7e293122d7ce with delete_type TABLET_DATA_TOMBSTONED (TS 5db2970f4bbd4a3c92d7d913f99716b1 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:51996
I20240501 08:00:08.653375 12777 tablet_replica.cc:330] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 08:00:08.653517 12777 raft_consensus.cc:2229] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:08.653657 12777 raft_consensus.cc:2258] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:08.654388 12777 ts_tablet_manager.cc:1875] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:08.656536 12777 ts_tablet_manager.cc:1888] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 08:00:08.656610 12777 log.cc:1196] T 6f24f629734649a3b31c7e293122d7ce P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/6f24f629734649a3b31c7e293122d7ce
I20240501 08:00:08.657086 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet 6f24f629734649a3b31c7e293122d7ce (table rebalance_test_table_1 [id=ad0b859435934b3c8fb86aff034e0daf]) successfully deleted
I20240501 08:00:08.669572 6593 consensus_queue.cc:237] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:08.671082 6462 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 08:00:08.671090 6527 consensus_peers.cc:485] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 -> Peer 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137:42389): Couldn't send request to peer 773008dcb81842c3bdf9ce259bb66b8a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6bf00ed2af0e42ca94c99a701fbe7dd1. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:08.671119 8030 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:08.671485 12821 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:08.671663 12154 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:08.673362 12820 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:08.673796 6462 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:08.673952 8030 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } } }
I20240501 08:00:08.677114 6059 catalog_manager.cc:5484] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 6 to 7, NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:08.678125 8161 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Refusing update from remote peer 29afadb287c049d9bb777d674b3697ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.678711 12655 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.682626 6593 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Refusing update from remote peer 29afadb287c049d9bb777d674b3697ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:08.683210 12423 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:08.689150 8030 consensus_queue.cc:237] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } }
I20240501 08:00:08.690286 6329 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer 29afadb287c049d9bb777d674b3697ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:08.690702 8161 raft_consensus.cc:1261] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Refusing update from remote peer 29afadb287c049d9bb777d674b3697ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:08.691038 12423 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.691319 12654 consensus_queue.cc:1035] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.693365 12654 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 29afadb287c049d9bb777d674b3697ce [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:08.693886 6331 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:08.694254 8161 raft_consensus.cc:2932] T 4e052973ae7744e8887d3df0d420f411 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:08.697175 6054 catalog_manager.cc:5484] T 4e052973ae7744e8887d3df0d420f411 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 8 to 10, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New cstate: current_term: 2 leader_uuid: "29afadb287c049d9bb777d674b3697ce" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } }
I20240501 08:00:08.697969 6045 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4e052973ae7744e8887d3df0d420f411 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:08.700239 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e052973ae7744e8887d3df0d420f411 with delete_type TABLET_DATA_TOMBSTONED (TS d2e209265a41425fba98c9d9b0551bd4 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:60522
I20240501 08:00:08.700627 12839 tablet_replica.cc:330] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: stopping tablet replica
I20240501 08:00:08.700811 12839 raft_consensus.cc:2229] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:08.700970 12839 raft_consensus.cc:2258] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:08.701669 12839 ts_tablet_manager.cc:1875] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:08.703900 12839 ts_tablet_manager.cc:1888] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 08:00:08.704010 12839 log.cc:1196] T 4e052973ae7744e8887d3df0d420f411 P d2e209265a41425fba98c9d9b0551bd4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/4e052973ae7744e8887d3df0d420f411
I20240501 08:00:08.704500 6046 catalog_manager.cc:4830] TS d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501): tablet 4e052973ae7744e8887d3df0d420f411 (table rebalance_test_table_2 [id=2992e99cfd4544b1923084358a23a41a]) successfully deleted
I20240501 08:00:08.793879 8161 consensus_queue.cc:237] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 3.10, Last appended by leader: 9, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } }
I20240501 08:00:08.795123 7900 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36 [term 3 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 10. Preceding OpId from leader: term: 3 index: 11. (index mismatch)
I20240501 08:00:08.795104 6198 raft_consensus.cc:1261] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 3 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 10. Preceding OpId from leader: term: 3 index: 11. (index mismatch)
I20240501 08:00:08.795490 12645 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:08.795715 12305 consensus_queue.cc:1035] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:08.797271 12300 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 [term 3 LEADER]: Committing config change with OpId 3.11: config changed from index 9 to 11, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:08.797719 7900 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P 7a80bdedf813425faf7617b4b07c9f36 [term 3 FOLLOWER]: Committing config change with OpId 3.11: config changed from index 9 to 11, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:08.797794 6198 raft_consensus.cc:2932] T 71bcfcbbdfc0491ea936168dd699fc92 P 953b945e3e7e4c71a61aa99c1208ab1d [term 3 FOLLOWER]: Committing config change with OpId 3.11: config changed from index 9 to 11, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } } }
I20240501 08:00:08.799202 6044 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 71bcfcbbdfc0491ea936168dd699fc92 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:08.800031 6051 catalog_manager.cc:5484] T 71bcfcbbdfc0491ea936168dd699fc92 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: config changed from index 9 to 11, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New cstate: current_term: 3 leader_uuid: "ef83010b77b441ffac2e41468a0a1bd1" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a80bdedf813425faf7617b4b07c9f36" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 39247 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:08.802346 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet 71bcfcbbdfc0491ea936168dd699fc92 with delete_type TABLET_DATA_TOMBSTONED (TS d2e209265a41425fba98c9d9b0551bd4 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:60522
I20240501 08:00:08.802618 12839 tablet_replica.cc:330] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4: stopping tablet replica
I20240501 08:00:08.802748 12839 raft_consensus.cc:2229] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:08.802836 12839 raft_consensus.cc:2258] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:08.803526 12839 ts_tablet_manager.cc:1875] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:08.805483 12839 ts_tablet_manager.cc:1888] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.10
I20240501 08:00:08.805546 12839 log.cc:1196] T 71bcfcbbdfc0491ea936168dd699fc92 P d2e209265a41425fba98c9d9b0551bd4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/71bcfcbbdfc0491ea936168dd699fc92
I20240501 08:00:08.805922 6046 catalog_manager.cc:4830] TS d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501): tablet 71bcfcbbdfc0491ea936168dd699fc92 (table rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]) successfully deleted
I20240501 08:00:08.840310 8030 consensus_queue.cc:237] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } }
I20240501 08:00:08.841483 8161 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Refusing update from remote peer 29afadb287c049d9bb777d674b3697ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:08.841490 6593 raft_consensus.cc:1261] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Refusing update from remote peer 29afadb287c049d9bb777d674b3697ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:08.841967 12655 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.842167 12654 consensus_queue.cc:1035] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:08.843734 12423 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P 29afadb287c049d9bb777d674b3697ce [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:08.844188 8161 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:08.844188 6593 raft_consensus.cc:2932] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:08.845175 6045 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet de48f8c2870f4517b5349c30a51fce56 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:08.846751 6054 catalog_manager.cc:5484] T de48f8c2870f4517b5349c30a51fce56 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 8 to 10, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 2 leader_uuid: "29afadb287c049d9bb777d674b3697ce" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } }
I20240501 08:00:08.849321 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet de48f8c2870f4517b5349c30a51fce56 with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:60224
I20240501 08:00:08.849831 12905 tablet_replica.cc:330] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 08:00:08.849979 12905 raft_consensus.cc:2229] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:08.850129 12905 raft_consensus.cc:2258] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:08.850673 12905 ts_tablet_manager.cc:1875] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:08.852427 12905 ts_tablet_manager.cc:1888] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 08:00:08.852488 12905 log.cc:1196] T de48f8c2870f4517b5349c30a51fce56 P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/de48f8c2870f4517b5349c30a51fce56
I20240501 08:00:08.852874 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet de48f8c2870f4517b5349c30a51fce56 (table rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]) successfully deleted
I20240501 08:00:09.064726 12966 ts_tablet_manager.cc:902] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a: Initiating tablet copy from peer d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501)
I20240501 08:00:09.065322 12966 tablet_copy_client.cc:308] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:42501
I20240501 08:00:09.065761 6614 tablet_copy_service.cc:140] P d2e209265a41425fba98c9d9b0551bd4: Received BeginTabletCopySession request for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 from peer 773008dcb81842c3bdf9ce259bb66b8a ({username='slave'} at 127.0.99.137:36345)
I20240501 08:00:09.065862 6614 tablet_copy_service.cc:161] P d2e209265a41425fba98c9d9b0551bd4: Beginning new tablet copy session on tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 from peer 773008dcb81842c3bdf9ce259bb66b8a at {username='slave'} at 127.0.99.137:36345: session id = 773008dcb81842c3bdf9ce259bb66b8a-6bf00ed2af0e42ca94c99a701fbe7dd1
I20240501 08:00:09.066618 6614 tablet_copy_source_session.cc:205] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:09.067400 12966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:09.069067 12966 tablet_copy_client.cc:783] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:09.069198 12966 tablet_copy_client.cc:647] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:09.070253 12970 tablet_copy_client.cc:1002] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:09.107139 12971 ts_tablet_manager.cc:902] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a: Initiating tablet copy from peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285)
I20240501 08:00:09.107604 12971 tablet_copy_client.cc:308] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:35285
I20240501 08:00:09.108124 6352 tablet_copy_service.cc:140] P 4a155eaa444d440d90fde420951458e4: Received BeginTabletCopySession request for tablet 88fa1c4aaf754e32bef123157b3bc62c from peer 773008dcb81842c3bdf9ce259bb66b8a ({username='slave'} at 127.0.99.137:35957)
I20240501 08:00:09.108230 6352 tablet_copy_service.cc:161] P 4a155eaa444d440d90fde420951458e4: Beginning new tablet copy session on tablet 88fa1c4aaf754e32bef123157b3bc62c from peer 773008dcb81842c3bdf9ce259bb66b8a at {username='slave'} at 127.0.99.137:35957: session id = 773008dcb81842c3bdf9ce259bb66b8a-88fa1c4aaf754e32bef123157b3bc62c
I20240501 08:00:09.108943 6352 tablet_copy_source_session.cc:205] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:09.109865 12971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88fa1c4aaf754e32bef123157b3bc62c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:09.111588 12971 tablet_copy_client.cc:783] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:09.111721 12971 tablet_copy_client.cc:647] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:09.112788 12975 tablet_copy_client.cc:1002] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Injecting latency into file download: 1500
I20240501 08:00:09.505053 12574 tablet_copy_client.cc:523] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:09.506314 12574 tablet_bootstrap.cc:492] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap starting.
I20240501 08:00:09.524123 12574 tablet_bootstrap.cc:492] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:09.524353 12574 tablet_bootstrap.cc:492] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap complete.
I20240501 08:00:09.524492 12574 ts_tablet_manager.cc:1367] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent bootstrapping tablet: real 0.018s user 0.019s sys 0.000s
I20240501 08:00:09.524855 12574 raft_consensus.cc:348] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:09.524963 12574 raft_consensus.cc:729] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 773008dcb81842c3bdf9ce259bb66b8a, State: Initialized, Role: LEARNER
I20240501 08:00:09.525050 12574 consensus_queue.cc:260] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:09.525305 12574 ts_tablet_manager.cc:1398] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:09.525878 8182 tablet_copy_service.cc:342] P ef83010b77b441ffac2e41468a0a1bd1: Request end of tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-96386dd2c62e45859e1f3b30510e8969 received from {username='slave'} at 127.0.99.137:54955
I20240501 08:00:09.525990 8182 tablet_copy_service.cc:433] P ef83010b77b441ffac2e41468a0a1bd1: ending tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-96386dd2c62e45859e1f3b30510e8969 on tablet 96386dd2c62e45859e1f3b30510e8969 with peer 773008dcb81842c3bdf9ce259bb66b8a
I20240501 08:00:09.532709 9188 raft_consensus.cc:1203] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 LEARNER]: Deduplicated request from leader. Original: 2.7->[2.8-2.8] Dedup: 2.8->[]
I20240501 08:00:09.563359 12581 tablet_copy_client.cc:523] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:09.564766 12581 tablet_bootstrap.cc:492] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap starting.
I20240501 08:00:09.583563 12581 tablet_bootstrap.cc:492] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:09.583853 12581 tablet_bootstrap.cc:492] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap complete.
I20240501 08:00:09.584051 12581 ts_tablet_manager.cc:1367] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent bootstrapping tablet: real 0.019s user 0.020s sys 0.000s
I20240501 08:00:09.584437 12581 raft_consensus.cc:348] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:09.584542 12581 raft_consensus.cc:729] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 773008dcb81842c3bdf9ce259bb66b8a, State: Initialized, Role: LEARNER
I20240501 08:00:09.584625 12581 consensus_queue.cc:260] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:09.584892 12581 ts_tablet_manager.cc:1398] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:09.585608 6220 tablet_copy_service.cc:342] P 953b945e3e7e4c71a61aa99c1208ab1d: Request end of tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-6b552a44c67940819f9ef64c68585eb1 received from {username='slave'} at 127.0.99.137:33301
I20240501 08:00:09.585695 6220 tablet_copy_service.cc:433] P 953b945e3e7e4c71a61aa99c1208ab1d: ending tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-6b552a44c67940819f9ef64c68585eb1 on tablet 6b552a44c67940819f9ef64c68585eb1 with peer 773008dcb81842c3bdf9ce259bb66b8a
I20240501 08:00:09.817693 12647 tablet_copy_client.cc:523] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:09.818986 12647 tablet_bootstrap.cc:492] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap starting.
I20240501 08:00:09.837258 12647 tablet_bootstrap.cc:492] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:09.837612 12647 tablet_bootstrap.cc:492] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap complete.
I20240501 08:00:09.837781 12647 ts_tablet_manager.cc:1367] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent bootstrapping tablet: real 0.019s user 0.018s sys 0.002s
I20240501 08:00:09.838224 12647 raft_consensus.cc:348] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:09.838294 9188 raft_consensus.cc:1203] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:09.838373 12647 raft_consensus.cc:729] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 773008dcb81842c3bdf9ce259bb66b8a, State: Initialized, Role: LEARNER
I20240501 08:00:09.838481 12647 consensus_queue.cc:260] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:09.838815 12647 ts_tablet_manager.cc:1398] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:09.839452 6220 tablet_copy_service.cc:342] P 953b945e3e7e4c71a61aa99c1208ab1d: Request end of tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-d929e6e09be943b68061ad4b1bb8e9f1 received from {username='slave'} at 127.0.99.137:33301
I20240501 08:00:09.839541 6220 tablet_copy_service.cc:433] P 953b945e3e7e4c71a61aa99c1208ab1d: ending tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-d929e6e09be943b68061ad4b1bb8e9f1 on tablet d929e6e09be943b68061ad4b1bb8e9f1 with peer 773008dcb81842c3bdf9ce259bb66b8a
I20240501 08:00:10.100852 9188 raft_consensus.cc:1203] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:10.126155 13086 raft_consensus.cc:1050] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1: attempting to promote NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a to VOTER
I20240501 08:00:10.126577 13086 consensus_queue.cc:237] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:10.127784 9189 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 LEARNER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:10.127961 6331 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:10.127977 6198 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 08:00:10.128271 13088 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:10.128468 13089 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:10.128643 13086 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:10.130411 13089 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.130831 9189 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.133087 6198 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.133174 6331 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.133381 6051 catalog_manager.cc:5484] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 reported cstate change: config changed from index 8 to 9, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "ef83010b77b441ffac2e41468a0a1bd1" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:10.137368 8161 consensus_queue.cc:237] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:10.138528 6331 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:10.138536 9189 raft_consensus.cc:1261] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 FOLLOWER]: Refusing update from remote peer ef83010b77b441ffac2e41468a0a1bd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:10.138892 12305 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:10.139334 12305 consensus_queue.cc:1035] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:10.140453 13086 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P ef83010b77b441ffac2e41468a0a1bd1 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.140843 6331 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.141301 9189 raft_consensus.cc:2932] T 96386dd2c62e45859e1f3b30510e8969 P 773008dcb81842c3bdf9ce259bb66b8a [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.141846 6044 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 96386dd2c62e45859e1f3b30510e8969 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:10.143450 6054 catalog_manager.cc:5484] T 96386dd2c62e45859e1f3b30510e8969 P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 9 to 10, VOTER 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "ef83010b77b441ffac2e41468a0a1bd1" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "ef83010b77b441ffac2e41468a0a1bd1" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 34937 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.145663 6181 tablet_service.cc:1514] Processing DeleteTablet for tablet 96386dd2c62e45859e1f3b30510e8969 with delete_type TABLET_DATA_TOMBSTONED (TS 953b945e3e7e4c71a61aa99c1208ab1d not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:45448
I20240501 08:00:10.146441 13097 tablet_replica.cc:330] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d: stopping tablet replica
I20240501 08:00:10.146579 13097 raft_consensus.cc:2229] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:10.146760 13097 raft_consensus.cc:2258] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:10.147444 13097 ts_tablet_manager.cc:1875] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:10.149257 13097 ts_tablet_manager.cc:1888] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 08:00:10.149322 13097 log.cc:1196] T 96386dd2c62e45859e1f3b30510e8969 P 953b945e3e7e4c71a61aa99c1208ab1d: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/96386dd2c62e45859e1f3b30510e8969
I20240501 08:00:10.149767 6046 catalog_manager.cc:4830] TS 953b945e3e7e4c71a61aa99c1208ab1d (127.0.99.129:44189): tablet 96386dd2c62e45859e1f3b30510e8969 (table rebalance_test_table_8 [id=e095e5bb38fa44f7b6f83be9c677435d]) successfully deleted
I20240501 08:00:10.251061 13145 raft_consensus.cc:1050] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d: attempting to promote NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a to VOTER
I20240501 08:00:10.251535 13145 consensus_queue.cc:237] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:10.253046 8294 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:10.253072 9189 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:10.253088 6331 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:10.253480 13144 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:10.253700 13145 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:10.254014 13147 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:10.256348 13145 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.256747 8294 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.257073 9189 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.258904 6331 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.259091 6051 catalog_manager.cc:5484] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:10.262166 6198 consensus_queue.cc:237] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:10.263381 8294 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:10.263485 9189 raft_consensus.cc:1261] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:10.263760 13147 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:10.263952 13144 consensus_queue.cc:1035] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:10.265651 13145 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.266237 9189 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.266762 8294 raft_consensus.cc:2932] T 6b552a44c67940819f9ef64c68585eb1 P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.267349 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6b552a44c67940819f9ef64c68585eb1 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:10.268388 6057 catalog_manager.cc:5484] T 6b552a44c67940819f9ef64c68585eb1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 8 to 9, VOTER 4a155eaa444d440d90fde420951458e4 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:10.270499 6311 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b552a44c67940819f9ef64c68585eb1 with delete_type TABLET_DATA_TOMBSTONED (TS 4a155eaa444d440d90fde420951458e4 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:60224
I20240501 08:00:10.271312 13151 tablet_replica.cc:330] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4: stopping tablet replica
I20240501 08:00:10.271445 13151 raft_consensus.cc:2229] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:10.271621 13151 raft_consensus.cc:2258] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:10.272217 13151 ts_tablet_manager.cc:1875] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:10.273995 13151 ts_tablet_manager.cc:1888] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 08:00:10.274055 13151 log.cc:1196] T 6b552a44c67940819f9ef64c68585eb1 P 4a155eaa444d440d90fde420951458e4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/6b552a44c67940819f9ef64c68585eb1
I20240501 08:00:10.274488 6046 catalog_manager.cc:4830] TS 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): tablet 6b552a44c67940819f9ef64c68585eb1 (table rebalance_test_table_6 [id=598165907b444443956cb026720c3e4c]) successfully deleted
I20240501 08:00:10.570570 12966 tablet_copy_client.cc:523] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:10.571954 12966 tablet_bootstrap.cc:492] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap starting.
I20240501 08:00:10.591261 12966 tablet_bootstrap.cc:492] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:10.591585 12966 tablet_bootstrap.cc:492] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap complete.
I20240501 08:00:10.591764 12966 ts_tablet_manager.cc:1367] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent bootstrapping tablet: real 0.020s user 0.016s sys 0.001s
I20240501 08:00:10.592238 12966 raft_consensus.cc:348] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:10.592353 12966 raft_consensus.cc:729] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 773008dcb81842c3bdf9ce259bb66b8a, State: Initialized, Role: LEARNER
I20240501 08:00:10.592451 12966 consensus_queue.cc:260] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:10.592729 12966 ts_tablet_manager.cc:1398] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 08:00:10.593214 6614 tablet_copy_service.cc:342] P d2e209265a41425fba98c9d9b0551bd4: Request end of tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-6bf00ed2af0e42ca94c99a701fbe7dd1 received from {username='slave'} at 127.0.99.137:36345
I20240501 08:00:10.593297 6614 tablet_copy_service.cc:433] P d2e209265a41425fba98c9d9b0551bd4: ending tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-6bf00ed2af0e42ca94c99a701fbe7dd1 on tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 with peer 773008dcb81842c3bdf9ce259bb66b8a
I20240501 08:00:10.613108 12971 tablet_copy_client.cc:523] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:10.614548 12971 tablet_bootstrap.cc:492] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap starting.
I20240501 08:00:10.635916 12971 tablet_bootstrap.cc:492] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:10.668124 12971 tablet_bootstrap.cc:492] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a: Bootstrap complete.
I20240501 08:00:10.668383 12971 ts_tablet_manager.cc:1367] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a: Time spent bootstrapping tablet: real 0.054s user 0.018s sys 0.005s
I20240501 08:00:10.668864 12971 raft_consensus.cc:348] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:10.669006 12971 raft_consensus.cc:729] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 773008dcb81842c3bdf9ce259bb66b8a, State: Initialized, Role: LEARNER
I20240501 08:00:10.669117 12971 consensus_queue.cc:260] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: true } }
I20240501 08:00:10.669425 12971 ts_tablet_manager.cc:1398] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:10.669881 6352 tablet_copy_service.cc:342] P 4a155eaa444d440d90fde420951458e4: Request end of tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-88fa1c4aaf754e32bef123157b3bc62c received from {username='slave'} at 127.0.99.137:35957
I20240501 08:00:10.669965 6352 tablet_copy_service.cc:433] P 4a155eaa444d440d90fde420951458e4: ending tablet copy session 773008dcb81842c3bdf9ce259bb66b8a-88fa1c4aaf754e32bef123157b3bc62c on tablet 88fa1c4aaf754e32bef123157b3bc62c with peer 773008dcb81842c3bdf9ce259bb66b8a
I20240501 08:00:10.718155 13144 raft_consensus.cc:1050] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d: attempting to promote NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a to VOTER
I20240501 08:00:10.718585 13144 consensus_queue.cc:237] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:10.719880 8030 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:10.720095 6462 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:10.720381 9188 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:10.720335 13147 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:10.720502 12060 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:10.721089 13147 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:10.722877 13145 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.723305 8030 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.724860 9188 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.725320 6462 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.726943 6054 catalog_manager.cc:5484] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:10.729607 6198 consensus_queue.cc:237] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:10.730923 9188 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:10.730994 8030 raft_consensus.cc:1261] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer 953b945e3e7e4c71a61aa99c1208ab1d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:10.731307 13147 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:10.731463 13144 consensus_queue.cc:1035] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:10.733181 13144 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.733595 8030 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.734063 9188 raft_consensus.cc:2932] T d929e6e09be943b68061ad4b1bb8e9f1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:10.734611 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d929e6e09be943b68061ad4b1bb8e9f1 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:10.736402 6051 catalog_manager.cc:5484] T d929e6e09be943b68061ad4b1bb8e9f1 P 953b945e3e7e4c71a61aa99c1208ab1d reported cstate change: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "953b945e3e7e4c71a61aa99c1208ab1d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:10.739029 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet d929e6e09be943b68061ad4b1bb8e9f1 with delete_type TABLET_DATA_TOMBSTONED (TS 5db2970f4bbd4a3c92d7d913f99716b1 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:51996
I20240501 08:00:10.739414 13209 tablet_replica.cc:330] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 08:00:10.739537 13209 raft_consensus.cc:2229] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:10.739712 13209 raft_consensus.cc:2258] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:10.740350 13209 ts_tablet_manager.cc:1875] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:10.742060 13209 ts_tablet_manager.cc:1888] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 08:00:10.742122 13209 log.cc:1196] T d929e6e09be943b68061ad4b1bb8e9f1 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/d929e6e09be943b68061ad4b1bb8e9f1
I20240501 08:00:10.742532 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet d929e6e09be943b68061ad4b1bb8e9f1 (table rebalance_test_table_3 [id=035e534d3b8747b0b2c93858aa7f89b0]) successfully deleted
I20240501 08:00:11.119036 9188 raft_consensus.cc:1203] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:11.266808 9188 raft_consensus.cc:1203] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 08:00:11.611476 13266 raft_consensus.cc:1050] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4: attempting to promote NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a to VOTER
I20240501 08:00:11.612002 13266 consensus_queue.cc:237] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:11.613806 9188 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:11.613871 8030 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:11.613896 6462 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:11.614285 13267 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:11.614493 13033 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:11.614662 13269 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:11.616480 13269 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.617019 8030 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.617136 9188 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.618561 6462 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.620538 6059 catalog_manager.cc:5484] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce reported cstate change: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5db2970f4bbd4a3c92d7d913f99716b1" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41487 } attrs { replace: true } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.624511 6593 consensus_queue.cc:237] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:11.625752 8030 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:11.625795 9188 raft_consensus.cc:1261] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Refusing update from remote peer d2e209265a41425fba98c9d9b0551bd4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:11.626180 13266 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:11.627161 13266 consensus_queue.cc:1035] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:11.627812 13267 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.628612 9188 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.629449 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:11.631021 6054 catalog_manager.cc:5484] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P d2e209265a41425fba98c9d9b0551bd4 reported cstate change: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "d2e209265a41425fba98c9d9b0551bd4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:11.631343 8030 raft_consensus.cc:2932] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 29afadb287c049d9bb777d674b3697ce [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } } peers { permanent_uuid: "29afadb287c049d9bb777d674b3697ce" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46293 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.633489 6441 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 with delete_type TABLET_DATA_TOMBSTONED (TS 5db2970f4bbd4a3c92d7d913f99716b1 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:51996
I20240501 08:00:11.633986 13277 tablet_replica.cc:330] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1: stopping tablet replica
I20240501 08:00:11.634195 13277 raft_consensus.cc:2229] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:11.634419 13277 raft_consensus.cc:2258] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:11.635313 13277 ts_tablet_manager.cc:1875] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:11.637967 13277 ts_tablet_manager.cc:1888] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 08:00:11.638048 13277 log.cc:1196] T 6bf00ed2af0e42ca94c99a701fbe7dd1 P 5db2970f4bbd4a3c92d7d913f99716b1: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/6bf00ed2af0e42ca94c99a701fbe7dd1
I20240501 08:00:11.638662 6046 catalog_manager.cc:4830] TS 5db2970f4bbd4a3c92d7d913f99716b1 (127.0.99.131:41487): tablet 6bf00ed2af0e42ca94c99a701fbe7dd1 (table rebalance_test_table_0 [id=55747160f09b44828631ac03da292061]) successfully deleted
I20240501 08:00:11.783262 11745 raft_consensus.cc:1050] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4: attempting to promote NON_VOTER 773008dcb81842c3bdf9ce259bb66b8a to VOTER
I20240501 08:00:11.783789 11745 consensus_queue.cc:237] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:11.785595 9188 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 LEARNER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:11.785758 6593 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:11.786046 13293 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:11.786188 8294 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:11.786270 13295 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:11.786752 13295 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:11.788895 13255 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.789243 6593 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.789243 8294 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.790997 9188 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.792258 6054 catalog_manager.cc:5484] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 7 to 8, 773008dcb81842c3bdf9ce259bb66b8a (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2e209265a41425fba98c9d9b0551bd4" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 42501 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:11.794858 6331 consensus_queue.cc:237] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } }
I20240501 08:00:11.796039 8294 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:11.796361 9188 raft_consensus.cc:1261] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Refusing update from remote peer 4a155eaa444d440d90fde420951458e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:11.796423 13293 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:11.796716 13295 consensus_queue.cc:1035] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:11.797870 13293 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.798547 8294 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 06c63579c7234194b5f6ef457dd5e3f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.798743 9188 raft_consensus.cc:2932] T 88fa1c4aaf754e32bef123157b3bc62c P 773008dcb81842c3bdf9ce259bb66b8a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } } }
I20240501 08:00:11.799321 6046 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 88fa1c4aaf754e32bef123157b3bc62c with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:11.800525 6057 catalog_manager.cc:5484] T 88fa1c4aaf754e32bef123157b3bc62c P 4a155eaa444d440d90fde420951458e4 reported cstate change: config changed from index 8 to 9, VOTER d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "4a155eaa444d440d90fde420951458e4" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4a155eaa444d440d90fde420951458e4" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06c63579c7234194b5f6ef457dd5e3f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 40449 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773008dcb81842c3bdf9ce259bb66b8a" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 42389 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:11.802851 6572 tablet_service.cc:1514] Processing DeleteTablet for tablet 88fa1c4aaf754e32bef123157b3bc62c with delete_type TABLET_DATA_TOMBSTONED (TS d2e209265a41425fba98c9d9b0551bd4 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:60522
I20240501 08:00:11.803313 13299 tablet_replica.cc:330] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4: stopping tablet replica
I20240501 08:00:11.803452 13299 raft_consensus.cc:2229] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:11.803637 13299 raft_consensus.cc:2258] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:11.804410 13299 ts_tablet_manager.cc:1875] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:11.806365 13299 ts_tablet_manager.cc:1888] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 08:00:11.806432 13299 log.cc:1196] T 88fa1c4aaf754e32bef123157b3bc62c P d2e209265a41425fba98c9d9b0551bd4: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/88fa1c4aaf754e32bef123157b3bc62c
I20240501 08:00:11.806934 6046 catalog_manager.cc:4830] TS d2e209265a41425fba98c9d9b0551bd4 (127.0.99.132:42501): tablet 88fa1c4aaf754e32bef123157b3bc62c (table rebalance_test_table_9 [id=18f76930631c430e8a058a048b6619e0]) successfully deleted
I20240501 08:00:12.037559 6311 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:12.038326 8274 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:12.040318 8011 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:12.045706 7880 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 08:00:12.053678 9169 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:12.054744 6181 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:12.058948 6572 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:12.068634 6441 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:12.091179 8142 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:12.184240 8274 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:12.184664 6311 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:12.187348 6441 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:12.188081 8011 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:12.188943 9169 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:12.208822 6181 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:12.217926 7880 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 08:00:12.221262 6572 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:12.222499 8142 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:12.488399 9169 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:12.489574 7880 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 08:00:12.491082 8011 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:12.492765 8274 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:12.497690 6441 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:12.498044 6311 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:12.500501 6572 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:12.513991 8142 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:12.517627 6181 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:12.644695 8274 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:12.647557 6311 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:12.651094 9169 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:12.664036 8011 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:12.664103 6441 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:12.664767 7880 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 08:00:12.673238 6181 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:12.677331 6572 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:12.694996 8142 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:12.820041 8011 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:12.821460 8274 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:12.828055 6441 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:12.829687 6181 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:12.831434 6311 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:12.834813 7880 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 08:00:12.837412 9169 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:12.858616 8142 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:12.868181 6572 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:12.976450 8274 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:12.976754 8011 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:12.978286 6311 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:12.982666 7880 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 08:00:12.986004 6441 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:13.001286 9169 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:13.001865 6181 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:13.018533 8142 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:13.019755 6572 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
b65b89b6d76f4286ad3d493da5bb629c | 127.0.99.190:41421 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:42393 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
06c63579c7234194b5f6ef457dd5e3f8 | 127.0.99.136:40449 | HEALTHY | <none> | 2 | 0
29afadb287c049d9bb777d674b3697ce | 127.0.99.134:46293 | HEALTHY | <none> | 2 | 0
4a155eaa444d440d90fde420951458e4 | 127.0.99.130:35285 | HEALTHY | <none> | 6 | 0
5db2970f4bbd4a3c92d7d913f99716b1 | 127.0.99.131:41487 | HEALTHY | <none> | 6 | 0
773008dcb81842c3bdf9ce259bb66b8a | 127.0.99.137:42389 | HEALTHY | <none> | 0 | 0
7a80bdedf813425faf7617b4b07c9f36 | 127.0.99.133:39247 | HEALTHY | <none> | 1 | 0
953b945e3e7e4c71a61aa99c1208ab1d | 127.0.99.129:44189 | HEALTHY | <none> | 5 | 0
d2e209265a41425fba98c9d9b0551bd4 | 127.0.99.132:42501 | HEALTHY | <none> | 4 | 0
ef83010b77b441ffac2e41468a0a1bd1 | 127.0.99.135:34937 | HEALTHY | <none> | 4 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 9
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
---------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 9 server(s) checked
local_ip_for_outbound_sockets | 127.0.99.129 | experimental | 127.0.99.129:44189
local_ip_for_outbound_sockets | 127.0.99.130 | experimental | 127.0.99.130:35285
local_ip_for_outbound_sockets | 127.0.99.131 | experimental | 127.0.99.131:41487
local_ip_for_outbound_sockets | 127.0.99.132 | experimental | 127.0.99.132:42501
local_ip_for_outbound_sockets | 127.0.99.133 | experimental | 127.0.99.133:39247
local_ip_for_outbound_sockets | 127.0.99.134 | experimental | 127.0.99.134:46293
local_ip_for_outbound_sockets | 127.0.99.135 | experimental | 127.0.99.135:34937
local_ip_for_outbound_sockets | 127.0.99.136 | experimental | 127.0.99.136:40449
local_ip_for_outbound_sockets | 127.0.99.137 | experimental | 127.0.99.137:42389
never_fsync | true | unsafe,advanced | all 9 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 9 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 9 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 9 server(s) checked
server_dump_info_format | pb | hidden | all 9 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.0.99.129:44189
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.0.99.130:35285
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.0.99.131:41487
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.0.99.132:42501
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.0.99.133:39247
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.0.99.134:46293
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.0.99.135:34937
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.0.99.136:40449
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/info.pb | hidden | 127.0.99.137:42389
tablet_copy_download_file_inject_latency_ms | 1500 | hidden | all 9 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:42393 | all 9 server(s) checked
time_source | builtin | all 9 server(s) checked
Version Summary
Version | Servers
-----------------+--------------------------
1.18.0-SNAPSHOT | all 10 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
rebalance_test_table_0 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_5 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_6 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_7 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_8 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_9 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 10
First Quartile | 10
Median | 10
Third Quartile | 10
Maximum | 10
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 9
Tables | 10
Tablets | 30
Replicas | 90
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240501 08:00:13.050781 398 log_verifier.cc:126] Checking tablet 16752f91ea894a7ba4017170c1dc6ec1
I20240501 08:00:13.077666 398 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 16752f91ea894a7ba4017170c1dc6ec1
I20240501 08:00:13.077876 398 log_verifier.cc:126] Checking tablet 2bbceded7f37434ca25b1068bdfc8ca1
I20240501 08:00:13.103703 398 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 2bbceded7f37434ca25b1068bdfc8ca1
I20240501 08:00:13.103780 398 log_verifier.cc:126] Checking tablet 3098236442b242b38c5937de465c8e1f
I20240501 08:00:13.128235 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 3098236442b242b38c5937de465c8e1f
I20240501 08:00:13.128299 398 log_verifier.cc:126] Checking tablet 33863ce53ec14aab9040f75e0360e0af
I20240501 08:00:13.156919 398 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 33863ce53ec14aab9040f75e0360e0af
I20240501 08:00:13.156989 398 log_verifier.cc:126] Checking tablet 41b9f1a284c34a9099cf1014db77f35c
I20240501 08:00:13.182976 398 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 41b9f1a284c34a9099cf1014db77f35c
I20240501 08:00:13.183053 398 log_verifier.cc:126] Checking tablet 47d3021e3ac74c6ba482b8b54066c36e
I20240501 08:00:13.208983 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 47d3021e3ac74c6ba482b8b54066c36e
I20240501 08:00:13.209054 398 log_verifier.cc:126] Checking tablet 4e052973ae7744e8887d3df0d420f411
I20240501 08:00:13.235457 398 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 4e052973ae7744e8887d3df0d420f411
I20240501 08:00:13.235543 398 log_verifier.cc:126] Checking tablet 4e60dad8ddd2402faae35be18662c8c3
I20240501 08:00:13.262075 398 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 4e60dad8ddd2402faae35be18662c8c3
I20240501 08:00:13.262156 398 log_verifier.cc:126] Checking tablet 5a31c7d3de834d1aab1ffa16e6df7e15
I20240501 08:00:13.286921 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 5a31c7d3de834d1aab1ffa16e6df7e15
I20240501 08:00:13.287009 398 log_verifier.cc:126] Checking tablet 5c411a11ec484cd5bd063e5fc3868e29
I20240501 08:00:13.312498 398 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 5c411a11ec484cd5bd063e5fc3868e29
I20240501 08:00:13.312578 398 log_verifier.cc:126] Checking tablet 5d0ae7c274af4ce1992e9507cbabc551
I20240501 08:00:13.337723 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 5d0ae7c274af4ce1992e9507cbabc551
I20240501 08:00:13.337802 398 log_verifier.cc:126] Checking tablet 5dae6c0b9c1d4009a93ed157e1f763c8
I20240501 08:00:13.363792 398 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 5dae6c0b9c1d4009a93ed157e1f763c8
I20240501 08:00:13.363881 398 log_verifier.cc:126] Checking tablet 6b552a44c67940819f9ef64c68585eb1
I20240501 08:00:13.391052 398 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 6b552a44c67940819f9ef64c68585eb1
I20240501 08:00:13.391131 398 log_verifier.cc:126] Checking tablet 6bf00ed2af0e42ca94c99a701fbe7dd1
I20240501 08:00:13.417650 398 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 6bf00ed2af0e42ca94c99a701fbe7dd1
I20240501 08:00:13.417729 398 log_verifier.cc:126] Checking tablet 6f24f629734649a3b31c7e293122d7ce
I20240501 08:00:13.442967 398 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 6f24f629734649a3b31c7e293122d7ce
I20240501 08:00:13.443045 398 log_verifier.cc:126] Checking tablet 71bcfcbbdfc0491ea936168dd699fc92
I20240501 08:00:13.468854 398 log_verifier.cc:177] Verified matching terms for 11 ops in tablet 71bcfcbbdfc0491ea936168dd699fc92
I20240501 08:00:13.468922 398 log_verifier.cc:126] Checking tablet 7a9dcf4944c34707a8cd0acebc8f9029
I20240501 08:00:13.493960 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 7a9dcf4944c34707a8cd0acebc8f9029
I20240501 08:00:13.494031 398 log_verifier.cc:126] Checking tablet 88fa1c4aaf754e32bef123157b3bc62c
I20240501 08:00:13.519706 398 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 88fa1c4aaf754e32bef123157b3bc62c
I20240501 08:00:13.519783 398 log_verifier.cc:126] Checking tablet 8bf587ac1d8141e99fceab28d89e4ea9
I20240501 08:00:13.545399 398 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 8bf587ac1d8141e99fceab28d89e4ea9
I20240501 08:00:13.545470 398 log_verifier.cc:126] Checking tablet 947b5d16224a42dcad709e7ddefc0b33
I20240501 08:00:13.569970 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 947b5d16224a42dcad709e7ddefc0b33
I20240501 08:00:13.570035 398 log_verifier.cc:126] Checking tablet 96386dd2c62e45859e1f3b30510e8969
I20240501 08:00:13.595986 398 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 96386dd2c62e45859e1f3b30510e8969
I20240501 08:00:13.596058 398 log_verifier.cc:126] Checking tablet 98d43e2703b14f9ebf926b5f2e2b2c56
I20240501 08:00:13.621460 398 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 98d43e2703b14f9ebf926b5f2e2b2c56
I20240501 08:00:13.621527 398 log_verifier.cc:126] Checking tablet a4b3aaeeb72e47e08c938b47310dd402
I20240501 08:00:13.646940 398 log_verifier.cc:177] Verified matching terms for 10 ops in tablet a4b3aaeeb72e47e08c938b47310dd402
I20240501 08:00:13.647012 398 log_verifier.cc:126] Checking tablet a97136a7017541eb8e4afb281c97e3cb
I20240501 08:00:13.673063 398 log_verifier.cc:177] Verified matching terms for 9 ops in tablet a97136a7017541eb8e4afb281c97e3cb
I20240501 08:00:13.673132 398 log_verifier.cc:126] Checking tablet d2a4fd236f394ba0ad1812c50b7e66a3
I20240501 08:00:13.698417 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet d2a4fd236f394ba0ad1812c50b7e66a3
I20240501 08:00:13.698493 398 log_verifier.cc:126] Checking tablet d929e6e09be943b68061ad4b1bb8e9f1
I20240501 08:00:13.724124 398 log_verifier.cc:177] Verified matching terms for 9 ops in tablet d929e6e09be943b68061ad4b1bb8e9f1
I20240501 08:00:13.724197 398 log_verifier.cc:126] Checking tablet dbc4b418e4c5493fa8a7734740225200
I20240501 08:00:13.752524 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet dbc4b418e4c5493fa8a7734740225200
I20240501 08:00:13.752656 398 log_verifier.cc:126] Checking tablet de48f8c2870f4517b5349c30a51fce56
I20240501 08:00:13.782954 398 log_verifier.cc:177] Verified matching terms for 10 ops in tablet de48f8c2870f4517b5349c30a51fce56
I20240501 08:00:13.783028 398 log_verifier.cc:126] Checking tablet e952704c6a3d43aba833484bba9fddbf
I20240501 08:00:13.813745 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet e952704c6a3d43aba833484bba9fddbf
I20240501 08:00:13.813823 398 log_verifier.cc:126] Checking tablet f4c62996a5a64d978850f2f7a31009f4
I20240501 08:00:13.844614 398 log_verifier.cc:177] Verified matching terms for 9 ops in tablet f4c62996a5a64d978850f2f7a31009f4
I20240501 08:00:13.845058 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 6119
I20240501 08:00:13.852480 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 6250
I20240501 08:00:13.859784 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 6381
I20240501 08:00:13.867233 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 6512
I20240501 08:00:13.874042 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 7818
W20240501 08:00:13.875866 8230 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.99.130:35285: connect: Connection refused (error 111)
W20240501 08:00:13.876811 8230 consensus_peers.cc:485] T 4e60dad8ddd2402faae35be18662c8c3 P 06c63579c7234194b5f6ef457dd5e3f8 -> Peer 4a155eaa444d440d90fde420951458e4 (127.0.99.130:35285): Couldn't send request to peer 4a155eaa444d440d90fde420951458e4. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.130:35285: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240501 08:00:13.880007 8096 connection.cc:535] client connection to 127.0.99.133:39247 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240501 08:00:13.880167 8096 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240501 08:00:13.880509 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 7949
W20240501 08:00:13.882444 8096 consensus_peers.cc:485] T a4b3aaeeb72e47e08c938b47310dd402 P ef83010b77b441ffac2e41468a0a1bd1 -> Peer 7a80bdedf813425faf7617b4b07c9f36 (127.0.99.133:39247): Couldn't send request to peer 7a80bdedf813425faf7617b4b07c9f36. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.133:39247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:13.886189 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 8080
I20240501 08:00:13.891858 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 8212
I20240501 08:00:13.897737 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 8976
I20240501 08:00:13.903156 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 6028
2024-05-01T08:00:13Z chronyd exiting
[ OK ] TserverAddedDuringRebalancingTest.TserverStarts/1 (29758 ms)
[----------] 1 test from TserverAddedDuringRebalancingTest (29758 ms total)
[----------] 1 test from RebalancerAndSingleReplicaTablets
[ RUN ] RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove/5
I20240501 08:00:13.993232 398 test_util.cc:268] Using random seed: -530569975
I20240501 08:00:13.994011 398 ts_itest-base.cc:112] Starting cluster with:
I20240501 08:00:13.994050 398 ts_itest-base.cc:113] --------------
I20240501 08:00:13.994060 398 ts_itest-base.cc:114] 4 tablet servers
I20240501 08:00:13.994069 398 ts_itest-base.cc:115] 1 replicas per TS
I20240501 08:00:13.994078 398 ts_itest-base.cc:116] --------------
2024-05-01T08:00:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T08:00:13Z Disabled control of system clock
I20240501 08:00:14.002439 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:42829
--webserver_interface=127.0.99.190
--webserver_port=0
--builtin_ntp_servers=127.0.99.148:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.99.190:42829
--raft_prepare_replacement_before_eviction=true
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false with env {}
W20240501 08:00:14.114373 13580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:14.114656 13580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:14.114718 13580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:14.118207 13580 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:14.118324 13580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 08:00:14.118359 13580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:14.118382 13580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 08:00:14.118412 13580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 08:00:14.122494 13580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.0.99.190:42829
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:42829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.99.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.13580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:14.123533 13580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:14.124817 13580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:14.133817 13585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:14.134256 13586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:14.135053 13588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:14.136185 13580 server_base.cc:1034] running on GCE node
I20240501 08:00:14.136793 13580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:14.137832 13580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:14.138991 13580 hybrid_clock.cc:648] HybridClock initialized: now 1714550414138976 us; error 38 us; skew 500 ppm
I20240501 08:00:14.141427 13580 webserver.cc:460] Webserver started at http://127.0.99.190:39607/ using document root <none> and password file <none>
I20240501 08:00:14.142058 13580 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:14.142124 13580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:14.142310 13580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:14.144174 13580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "9aea89d8f45c4585aeb7849d0b1c863e"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:14.144675 13580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "9aea89d8f45c4585aeb7849d0b1c863e"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:14.148284 13580 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:14.150839 13594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:14.152068 13580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 08:00:14.152211 13580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
uuid: "9aea89d8f45c4585aeb7849d0b1c863e"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:14.152313 13580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:14.171757 13580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:14.172542 13580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:14.172714 13580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:14.180025 13580 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:42829
I20240501 08:00:14.180030 13646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:42829 every 8 connection(s)
I20240501 08:00:14.181087 13580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 08:00:14.183904 13647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:14.189133 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 13580
I20240501 08:00:14.189299 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 08:00:14.189607 13647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e: Bootstrap starting.
I20240501 08:00:14.192169 13647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:14.192888 13647 log.cc:824] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:14.194667 13647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e: No bootstrap required, opened a new log
I20240501 08:00:14.197579 13647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aea89d8f45c4585aeb7849d0b1c863e" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 42829 } }
I20240501 08:00:14.197820 13647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:14.197878 13647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aea89d8f45c4585aeb7849d0b1c863e, State: Initialized, Role: FOLLOWER
I20240501 08:00:14.198311 13647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aea89d8f45c4585aeb7849d0b1c863e" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 42829 } }
I20240501 08:00:14.198444 13647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:14.198504 13647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:14.198613 13647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:14.199499 13647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aea89d8f45c4585aeb7849d0b1c863e" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 42829 } }
I20240501 08:00:14.199819 13647 leader_election.cc:302] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9aea89d8f45c4585aeb7849d0b1c863e; no voters:
I20240501 08:00:14.200147 13647 leader_election.cc:288] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:14.200536 13652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:14.200722 13652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [term 1 LEADER]: Becoming Leader. State: Replica: 9aea89d8f45c4585aeb7849d0b1c863e, State: Running, Role: LEADER
I20240501 08:00:14.200987 13652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aea89d8f45c4585aeb7849d0b1c863e" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 42829 } }
I20240501 08:00:14.201211 13647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [sys.catalog]: configured and running, proceeding with master startup.
I20240501 08:00:14.202550 13653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9aea89d8f45c4585aeb7849d0b1c863e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aea89d8f45c4585aeb7849d0b1c863e" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 42829 } } }
I20240501 08:00:14.202525 13654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9aea89d8f45c4585aeb7849d0b1c863e. Latest consensus state: current_term: 1 leader_uuid: "9aea89d8f45c4585aeb7849d0b1c863e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aea89d8f45c4585aeb7849d0b1c863e" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 42829 } } }
I20240501 08:00:14.202669 13653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [sys.catalog]: This master's current role is: LEADER
I20240501 08:00:14.202673 13654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e [sys.catalog]: This master's current role is: LEADER
I20240501 08:00:14.203040 13663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 08:00:14.206126 13663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 08:00:14.211509 13663 catalog_manager.cc:1347] Generated new cluster ID: 1ac05cb2a21049aebaa8b797b6a4580f
I20240501 08:00:14.211611 13663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 08:00:14.227639 13663 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 08:00:14.228787 13663 catalog_manager.cc:1504] Loading token signing keys...
I20240501 08:00:14.238185 13663 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 9aea89d8f45c4585aeb7849d0b1c863e: Generated new TSK 0
I20240501 08:00:14.238988 13663 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 08:00:14.247280 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.129:0
--local_ip_for_outbound_sockets=127.0.99.129
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:42829
--builtin_ntp_servers=127.0.99.148:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 08:00:14.378412 13671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:14.378680 13671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:14.378744 13671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:14.382066 13671 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:14.382280 13671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:14.382505 13671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.129
I20240501 08:00:14.386644 13671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:42829
--never_fsync=true
--heap_profile_path=/tmp/kudu.13671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.129
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:14.387648 13671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:14.388897 13671 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:14.398344 13677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:14.398849 13679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:14.399557 13676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:14.400112 13671 server_base.cc:1034] running on GCE node
I20240501 08:00:14.400604 13671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:14.401230 13671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:14.402401 13671 hybrid_clock.cc:648] HybridClock initialized: now 1714550414402380 us; error 42 us; skew 500 ppm
I20240501 08:00:14.404848 13671 webserver.cc:460] Webserver started at http://127.0.99.129:42141/ using document root <none> and password file <none>
I20240501 08:00:14.405434 13671 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:14.405493 13671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:14.405684 13671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:14.407495 13671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "d25ceaf478984a92871988a205e1db84"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:14.408071 13671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "d25ceaf478984a92871988a205e1db84"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:14.412129 13671 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20240501 08:00:14.414732 13685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:14.416150 13671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:14.416286 13671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "d25ceaf478984a92871988a205e1db84"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:14.416394 13671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:14.432794 13671 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:14.433485 13671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:14.433672 13671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:14.434252 13671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:14.435660 13671 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:14.435756 13671 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:14.435866 13671 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:14.435914 13671 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:14.449237 13671 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:37611
I20240501 08:00:14.449254 13798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:37611 every 8 connection(s)
I20240501 08:00:14.450769 13671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 08:00:14.454864 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 13671
I20240501 08:00:14.455022 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 08:00:14.457840 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:0
--local_ip_for_outbound_sockets=127.0.99.130
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:42829
--builtin_ntp_servers=127.0.99.148:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:14.463518 13799 heartbeater.cc:347] Connected to a master server at 127.0.99.190:42829
I20240501 08:00:14.463804 13799 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:14.464576 13799 heartbeater.cc:511] Master 127.0.99.190:42829 requested a full tablet report, sending...
I20240501 08:00:14.466434 13611 ts_manager.cc:194] Registered new tserver with Master: d25ceaf478984a92871988a205e1db84 (127.0.99.129:37611)
I20240501 08:00:14.468294 13611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.129:54007
W20240501 08:00:14.577873 13802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:14.578140 13802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:14.578189 13802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:14.581434 13802 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:14.581517 13802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:14.581586 13802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240501 08:00:14.586390 13802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:42829
--never_fsync=true
--heap_profile_path=/tmp/kudu.13802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:14.587389 13802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:14.588622 13802 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:14.598238 13807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:14.599159 13808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:14.599598 13810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:14.600194 13802 server_base.cc:1034] running on GCE node
I20240501 08:00:14.600931 13802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:14.601644 13802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:14.602847 13802 hybrid_clock.cc:648] HybridClock initialized: now 1714550414602813 us; error 55 us; skew 500 ppm
I20240501 08:00:14.605350 13802 webserver.cc:460] Webserver started at http://127.0.99.130:38361/ using document root <none> and password file <none>
I20240501 08:00:14.606022 13802 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:14.606086 13802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:14.606277 13802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:14.608103 13802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "10c3e6b8e0ca438895a09b8f6aee5e03"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:14.608585 13802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "10c3e6b8e0ca438895a09b8f6aee5e03"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:14.612185 13802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:14.614607 13816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:14.615780 13802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:14.615906 13802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "10c3e6b8e0ca438895a09b8f6aee5e03"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:14.616039 13802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:14.643076 13802 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:14.643764 13802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:14.643961 13802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:14.644553 13802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:14.645505 13802 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:14.645556 13802 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:14.645608 13802 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:14.645642 13802 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:14.654340 13802 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:33271
I20240501 08:00:14.654353 13929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:33271 every 8 connection(s)
I20240501 08:00:14.655365 13802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 08:00:14.664788 13930 heartbeater.cc:347] Connected to a master server at 127.0.99.190:42829
I20240501 08:00:14.665060 13930 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:14.665602 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 13802
I20240501 08:00:14.665728 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 08:00:14.665791 13930 heartbeater.cc:511] Master 127.0.99.190:42829 requested a full tablet report, sending...
I20240501 08:00:14.666936 13611 ts_manager.cc:194] Registered new tserver with Master: 10c3e6b8e0ca438895a09b8f6aee5e03 (127.0.99.130:33271)
I20240501 08:00:14.667500 13611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:48475
I20240501 08:00:14.668285 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:0
--local_ip_for_outbound_sockets=127.0.99.131
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:42829
--builtin_ntp_servers=127.0.99.148:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 08:00:14.785130 13933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:14.785410 13933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:14.785478 13933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:14.788869 13933 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:14.788969 13933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:14.789067 13933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240501 08:00:14.793177 13933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:42829
--never_fsync=true
--heap_profile_path=/tmp/kudu.13933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:14.794301 13933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:14.795779 13933 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:14.805441 13941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:14.805979 13938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:14.806659 13939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:14.807380 13933 server_base.cc:1034] running on GCE node
I20240501 08:00:14.807901 13933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:14.808625 13933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:14.809814 13933 hybrid_clock.cc:648] HybridClock initialized: now 1714550414809795 us; error 41 us; skew 500 ppm
I20240501 08:00:14.812057 13933 webserver.cc:460] Webserver started at http://127.0.99.131:34113/ using document root <none> and password file <none>
I20240501 08:00:14.812669 13933 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:14.812736 13933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:14.812918 13933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:14.814675 13933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "311f3bbbafd4430994bf923051c9f78a"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:14.815176 13933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "311f3bbbafd4430994bf923051c9f78a"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:14.818907 13933 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20240501 08:00:14.821520 13947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:14.822674 13933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:14.822850 13933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "311f3bbbafd4430994bf923051c9f78a"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:14.822955 13933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:14.846103 13933 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:14.846918 13933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:14.847106 13933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:14.847679 13933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:14.848695 13933 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:14.848748 13933 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:14.848805 13933 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:14.848834 13933 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:14.858080 13933 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:37069
I20240501 08:00:14.858094 14060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:37069 every 8 connection(s)
I20240501 08:00:14.859241 13933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 08:00:14.865324 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 13933
I20240501 08:00:14.865453 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 08:00:14.868090 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:0
--local_ip_for_outbound_sockets=127.0.99.132
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:42829
--builtin_ntp_servers=127.0.99.148:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:14.868947 14061 heartbeater.cc:347] Connected to a master server at 127.0.99.190:42829
I20240501 08:00:14.869210 14061 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:14.869833 14061 heartbeater.cc:511] Master 127.0.99.190:42829 requested a full tablet report, sending...
I20240501 08:00:14.870970 13611 ts_manager.cc:194] Registered new tserver with Master: 311f3bbbafd4430994bf923051c9f78a (127.0.99.131:37069)
I20240501 08:00:14.871560 13611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:43397
W20240501 08:00:14.989796 14064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:14.990108 14064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:14.990175 14064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:14.993921 14064 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:14.994025 14064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:14.994122 14064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240501 08:00:14.998286 14064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:42829
--never_fsync=true
--heap_profile_path=/tmp/kudu.14064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:14.999305 14064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:15.000533 14064 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:15.009604 14069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:15.010284 14072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:15.011221 14070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:15.012300 14064 server_base.cc:1034] running on GCE node
I20240501 08:00:15.012811 14064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:15.013432 14064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:15.014617 14064 hybrid_clock.cc:648] HybridClock initialized: now 1714550415014595 us; error 39 us; skew 500 ppm
I20240501 08:00:15.016938 14064 webserver.cc:460] Webserver started at http://127.0.99.132:45077/ using document root <none> and password file <none>
I20240501 08:00:15.017527 14064 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:15.017601 14064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:15.017800 14064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:15.019604 14064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "d44a3d2090f9401f91781f9404d58f04"
format_stamp: "Formatted at 2024-05-01 08:00:15 on dist-test-slave-f81v"
I20240501 08:00:15.020148 14064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "d44a3d2090f9401f91781f9404d58f04"
format_stamp: "Formatted at 2024-05-01 08:00:15 on dist-test-slave-f81v"
I20240501 08:00:15.023798 14064 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240501 08:00:15.026438 14078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:15.027873 14064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:15.028067 14064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "d44a3d2090f9401f91781f9404d58f04"
format_stamp: "Formatted at 2024-05-01 08:00:15 on dist-test-slave-f81v"
I20240501 08:00:15.028182 14064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:15.064007 14064 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:15.064857 14064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:15.065047 14064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:15.065657 14064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:15.066624 14064 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:15.066679 14064 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:15.066732 14064 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:15.066758 14064 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:15.075785 14064 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:45141
I20240501 08:00:15.075805 14191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:45141 every 8 connection(s)
I20240501 08:00:15.076849 14064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 08:00:15.085656 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 14064
I20240501 08:00:15.085844 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 08:00:15.086560 14192 heartbeater.cc:347] Connected to a master server at 127.0.99.190:42829
I20240501 08:00:15.086920 14192 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:15.087654 14192 heartbeater.cc:511] Master 127.0.99.190:42829 requested a full tablet report, sending...
I20240501 08:00:15.088842 13611 ts_manager.cc:194] Registered new tserver with Master: d44a3d2090f9401f91781f9404d58f04 (127.0.99.132:45141)
I20240501 08:00:15.089479 13611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:37765
I20240501 08:00:15.102069 398 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240501 08:00:15.112610 13611 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57542:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240501 08:00:15.130545 13733 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab30dbad3fd4dca9273431161444df1 (DEFAULT_TABLE table=TestTable [id=76b466f4dd384f11ae5fd2a4feba71a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:15.131587 13733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab30dbad3fd4dca9273431161444df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:15.136930 14205 tablet_bootstrap.cc:492] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84: Bootstrap starting.
I20240501 08:00:15.138906 14205 tablet_bootstrap.cc:654] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:15.139595 14205 log.cc:824] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:15.141690 14205 tablet_bootstrap.cc:492] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84: No bootstrap required, opened a new log
I20240501 08:00:15.141914 14205 ts_tablet_manager.cc:1367] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20240501 08:00:15.146078 14205 raft_consensus.cc:348] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:15.146399 14205 raft_consensus.cc:374] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:15.146458 14205 raft_consensus.cc:729] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25ceaf478984a92871988a205e1db84, State: Initialized, Role: FOLLOWER
I20240501 08:00:15.146982 14205 consensus_queue.cc:260] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:15.147151 14205 raft_consensus.cc:388] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:15.147218 14205 raft_consensus.cc:482] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:15.147337 14205 raft_consensus.cc:3037] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:15.148488 14205 raft_consensus.cc:504] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:15.148854 14205 leader_election.cc:302] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d25ceaf478984a92871988a205e1db84; no voters:
I20240501 08:00:15.149166 14205 leader_election.cc:288] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:15.149315 14207 raft_consensus.cc:2781] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:15.149544 14207 raft_consensus.cc:686] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [term 1 LEADER]: Becoming Leader. State: Replica: d25ceaf478984a92871988a205e1db84, State: Running, Role: LEADER
I20240501 08:00:15.149987 14207 consensus_queue.cc:237] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:15.150130 14205 ts_tablet_manager.cc:1398] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84: Time spent starting tablet: real 0.008s user 0.008s sys 0.001s
I20240501 08:00:15.150983 13799 heartbeater.cc:503] Master 127.0.99.190:42829 was elected leader, sending a full tablet report...
I20240501 08:00:15.154233 13611 catalog_manager.cc:5484] T 7ab30dbad3fd4dca9273431161444df1 P d25ceaf478984a92871988a205e1db84 reported cstate change: term changed from 0 to 1, leader changed from <none> to d25ceaf478984a92871988a205e1db84 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "d25ceaf478984a92871988a205e1db84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:15.168087 398 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240501 08:00:15.169624 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver d25ceaf478984a92871988a205e1db84 to finish bootstrapping
I20240501 08:00:15.175485 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 10c3e6b8e0ca438895a09b8f6aee5e03 to finish bootstrapping
I20240501 08:00:15.182783 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 311f3bbbafd4430994bf923051c9f78a to finish bootstrapping
I20240501 08:00:15.190227 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver d44a3d2090f9401f91781f9404d58f04 to finish bootstrapping
I20240501 08:00:15.198745 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 13933
I20240501 08:00:15.204944 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 14064
I20240501 08:00:15.670177 13930 heartbeater.cc:503] Master 127.0.99.190:42829 was elected leader, sending a full tablet report...
I20240501 08:00:18.961371 13611 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57542:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 08:00:18.968691 13733 tablet_service.cc:1467] Processing CreateTablet for tablet 15b80f759a8e4e3783257b7e66667a5a (DEFAULT_TABLE table=rebalance_test_table_0 [id=299c17e9ce164f3c8a10ad9395041796]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:18.969060 13733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15b80f759a8e4e3783257b7e66667a5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:18.971356 14222 tablet_bootstrap.cc:492] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84: Bootstrap starting.
I20240501 08:00:18.972278 14222 tablet_bootstrap.cc:654] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:18.973470 14222 tablet_bootstrap.cc:492] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84: No bootstrap required, opened a new log
I20240501 08:00:18.973592 14222 ts_tablet_manager.cc:1367] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:18.974159 14222 raft_consensus.cc:348] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:18.974268 14222 raft_consensus.cc:374] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:18.974300 14222 raft_consensus.cc:729] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25ceaf478984a92871988a205e1db84, State: Initialized, Role: FOLLOWER
I20240501 08:00:18.974031 13863 tablet_service.cc:1467] Processing CreateTablet for tablet ed20fa3af025445fbb10e0beac7c6da9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=299c17e9ce164f3c8a10ad9395041796]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:18.974452 14222 consensus_queue.cc:260] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:18.974020 13864 tablet_service.cc:1467] Processing CreateTablet for tablet 3d8ae6217d9b450cb07b930620fc5af0 (DEFAULT_TABLE table=rebalance_test_table_0 [id=299c17e9ce164f3c8a10ad9395041796]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:18.974524 14222 raft_consensus.cc:388] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:18.974575 14222 raft_consensus.cc:482] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:18.974639 14222 raft_consensus.cc:3037] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:18.975133 13864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d8ae6217d9b450cb07b930620fc5af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:18.975416 14222 raft_consensus.cc:504] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:18.975502 13863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed20fa3af025445fbb10e0beac7c6da9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:18.975543 14222 leader_election.cc:302] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d25ceaf478984a92871988a205e1db84; no voters:
I20240501 08:00:18.975694 14222 leader_election.cc:288] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:18.975899 14225 raft_consensus.cc:2781] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:18.976015 14222 ts_tablet_manager.cc:1398] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:18.976167 14225 raft_consensus.cc:686] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [term 1 LEADER]: Becoming Leader. State: Replica: d25ceaf478984a92871988a205e1db84, State: Running, Role: LEADER
I20240501 08:00:18.976334 14225 consensus_queue.cc:237] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:18.978139 13611 catalog_manager.cc:5484] T 15b80f759a8e4e3783257b7e66667a5a P d25ceaf478984a92871988a205e1db84 reported cstate change: term changed from 0 to 1, leader changed from <none> to d25ceaf478984a92871988a205e1db84 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "d25ceaf478984a92871988a205e1db84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:18.981655 14229 tablet_bootstrap.cc:492] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03: Bootstrap starting.
I20240501 08:00:18.983798 14229 tablet_bootstrap.cc:654] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:18.984577 14229 log.cc:824] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:18.986585 14229 tablet_bootstrap.cc:492] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03: No bootstrap required, opened a new log
I20240501 08:00:18.986783 14229 ts_tablet_manager.cc:1367] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240501 08:00:18.989868 14229 raft_consensus.cc:348] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:18.990085 14229 raft_consensus.cc:374] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:18.990132 14229 raft_consensus.cc:729] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c3e6b8e0ca438895a09b8f6aee5e03, State: Initialized, Role: FOLLOWER
I20240501 08:00:18.990545 14229 consensus_queue.cc:260] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:18.990664 14229 raft_consensus.cc:388] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:18.990708 14229 raft_consensus.cc:482] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:18.990813 14229 raft_consensus.cc:3037] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:18.991741 14229 raft_consensus.cc:504] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:18.992118 14229 leader_election.cc:302] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10c3e6b8e0ca438895a09b8f6aee5e03; no voters:
I20240501 08:00:18.992445 14229 leader_election.cc:288] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:18.992537 14231 raft_consensus.cc:2781] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:18.992756 14231 raft_consensus.cc:686] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 LEADER]: Becoming Leader. State: Replica: 10c3e6b8e0ca438895a09b8f6aee5e03, State: Running, Role: LEADER
I20240501 08:00:18.993357 14229 ts_tablet_manager.cc:1398] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20240501 08:00:18.993371 14231 consensus_queue.cc:237] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:18.993685 14229 tablet_bootstrap.cc:492] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03: Bootstrap starting.
I20240501 08:00:18.994731 14229 tablet_bootstrap.cc:654] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:18.995990 14229 tablet_bootstrap.cc:492] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03: No bootstrap required, opened a new log
I20240501 08:00:18.996089 14229 ts_tablet_manager.cc:1367] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:18.996572 14229 raft_consensus.cc:348] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:18.996659 14229 raft_consensus.cc:374] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:18.996688 14229 raft_consensus.cc:729] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c3e6b8e0ca438895a09b8f6aee5e03, State: Initialized, Role: FOLLOWER
I20240501 08:00:18.996810 14229 consensus_queue.cc:260] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:18.996768 13611 catalog_manager.cc:5484] T ed20fa3af025445fbb10e0beac7c6da9 P 10c3e6b8e0ca438895a09b8f6aee5e03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10c3e6b8e0ca438895a09b8f6aee5e03 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:18.996886 14229 raft_consensus.cc:388] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:18.996927 14229 raft_consensus.cc:482] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:18.996973 14229 raft_consensus.cc:3037] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:18.997674 14229 raft_consensus.cc:504] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:18.997797 14229 leader_election.cc:302] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10c3e6b8e0ca438895a09b8f6aee5e03; no voters:
I20240501 08:00:18.997887 14229 leader_election.cc:288] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:18.997967 14233 raft_consensus.cc:2781] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:18.998059 14233 raft_consensus.cc:686] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 LEADER]: Becoming Leader. State: Replica: 10c3e6b8e0ca438895a09b8f6aee5e03, State: Running, Role: LEADER
I20240501 08:00:18.998116 14229 ts_tablet_manager.cc:1398] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240501 08:00:18.998158 14233 consensus_queue.cc:237] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.000358 13611 catalog_manager.cc:5484] T 3d8ae6217d9b450cb07b930620fc5af0 P 10c3e6b8e0ca438895a09b8f6aee5e03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10c3e6b8e0ca438895a09b8f6aee5e03 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:19.180186 13611 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57542:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 08:00:19.185833 13733 tablet_service.cc:1467] Processing CreateTablet for tablet b5ed715f9be94deda847ab066a0d90bd (DEFAULT_TABLE table=rebalance_test_table_1 [id=56f6a9e35b2e4c41ab0b20a645fbf051]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:19.186156 13733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5ed715f9be94deda847ab066a0d90bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:19.186081 13864 tablet_service.cc:1467] Processing CreateTablet for tablet 0ebf15eda0fa406eb2bf307a83c28f37 (DEFAULT_TABLE table=rebalance_test_table_1 [id=56f6a9e35b2e4c41ab0b20a645fbf051]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:19.186503 13864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ebf15eda0fa406eb2bf307a83c28f37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:19.188228 14222 tablet_bootstrap.cc:492] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84: Bootstrap starting.
I20240501 08:00:19.188769 13733 tablet_service.cc:1467] Processing CreateTablet for tablet 6404cb72b6b0473c837a1383cecaa197 (DEFAULT_TABLE table=rebalance_test_table_1 [id=56f6a9e35b2e4c41ab0b20a645fbf051]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:19.188992 13733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6404cb72b6b0473c837a1383cecaa197. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:19.189385 14222 tablet_bootstrap.cc:654] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:19.189949 14229 tablet_bootstrap.cc:492] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03: Bootstrap starting.
I20240501 08:00:19.190682 14229 tablet_bootstrap.cc:654] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:19.191089 14222 tablet_bootstrap.cc:492] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84: No bootstrap required, opened a new log
I20240501 08:00:19.191161 14222 ts_tablet_manager.cc:1367] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:19.191597 14222 raft_consensus.cc:348] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.191694 14222 raft_consensus.cc:374] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:19.191720 14222 raft_consensus.cc:729] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25ceaf478984a92871988a205e1db84, State: Initialized, Role: FOLLOWER
I20240501 08:00:19.191850 14222 consensus_queue.cc:260] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.191955 14222 raft_consensus.cc:388] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:19.191991 14222 raft_consensus.cc:482] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:19.192028 14222 raft_consensus.cc:3037] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:19.192530 14229 tablet_bootstrap.cc:492] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03: No bootstrap required, opened a new log
I20240501 08:00:19.192616 14229 ts_tablet_manager.cc:1367] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:19.192756 14222 raft_consensus.cc:504] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.192879 14222 leader_election.cc:302] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d25ceaf478984a92871988a205e1db84; no voters:
I20240501 08:00:19.193027 14222 leader_election.cc:288] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:19.193042 14229 raft_consensus.cc:348] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.193114 14226 raft_consensus.cc:2781] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:19.193145 14229 raft_consensus.cc:374] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:19.193183 14229 raft_consensus.cc:729] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c3e6b8e0ca438895a09b8f6aee5e03, State: Initialized, Role: FOLLOWER
I20240501 08:00:19.193207 14226 raft_consensus.cc:686] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [term 1 LEADER]: Becoming Leader. State: Replica: d25ceaf478984a92871988a205e1db84, State: Running, Role: LEADER
I20240501 08:00:19.193310 14229 consensus_queue.cc:260] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.193363 14222 ts_tablet_manager.cc:1398] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240501 08:00:19.193329 14226 consensus_queue.cc:237] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.193392 14229 raft_consensus.cc:388] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:19.193459 14229 raft_consensus.cc:482] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:19.193497 14229 raft_consensus.cc:3037] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:19.193815 14222 tablet_bootstrap.cc:492] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84: Bootstrap starting.
I20240501 08:00:19.194972 14222 tablet_bootstrap.cc:654] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:19.195356 14229 raft_consensus.cc:504] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.195495 14229 leader_election.cc:302] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10c3e6b8e0ca438895a09b8f6aee5e03; no voters:
I20240501 08:00:19.195643 14233 raft_consensus.cc:2781] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:19.195722 14233 raft_consensus.cc:686] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 LEADER]: Becoming Leader. State: Replica: 10c3e6b8e0ca438895a09b8f6aee5e03, State: Running, Role: LEADER
I20240501 08:00:19.195839 14233 consensus_queue.cc:237] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.195868 13611 catalog_manager.cc:5484] T b5ed715f9be94deda847ab066a0d90bd P d25ceaf478984a92871988a205e1db84 reported cstate change: term changed from 0 to 1, leader changed from <none> to d25ceaf478984a92871988a205e1db84 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "d25ceaf478984a92871988a205e1db84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:19.196081 14222 tablet_bootstrap.cc:492] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84: No bootstrap required, opened a new log
I20240501 08:00:19.196146 14222 ts_tablet_manager.cc:1367] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 08:00:19.196336 14229 leader_election.cc:288] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:19.196534 14222 raft_consensus.cc:348] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.196621 14222 raft_consensus.cc:374] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:19.196648 14222 raft_consensus.cc:729] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25ceaf478984a92871988a205e1db84, State: Initialized, Role: FOLLOWER
I20240501 08:00:19.196767 14222 consensus_queue.cc:260] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.196842 14222 raft_consensus.cc:388] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:19.196871 14222 raft_consensus.cc:482] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:19.196907 14222 raft_consensus.cc:3037] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:19.197715 14222 raft_consensus.cc:504] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.197845 14222 leader_election.cc:302] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d25ceaf478984a92871988a205e1db84; no voters:
I20240501 08:00:19.197928 14222 leader_election.cc:288] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:19.198014 14229 ts_tablet_manager.cc:1398] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20240501 08:00:19.198112 14222 ts_tablet_manager.cc:1398] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:19.198611 13610 catalog_manager.cc:5484] T 0ebf15eda0fa406eb2bf307a83c28f37 P 10c3e6b8e0ca438895a09b8f6aee5e03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10c3e6b8e0ca438895a09b8f6aee5e03 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:19.199173 14225 raft_consensus.cc:2781] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:19.199262 14225 raft_consensus.cc:686] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [term 1 LEADER]: Becoming Leader. State: Replica: d25ceaf478984a92871988a205e1db84, State: Running, Role: LEADER
I20240501 08:00:19.199370 14225 consensus_queue.cc:237] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.200851 13611 catalog_manager.cc:5484] T 6404cb72b6b0473c837a1383cecaa197 P d25ceaf478984a92871988a205e1db84 reported cstate change: term changed from 0 to 1, leader changed from <none> to d25ceaf478984a92871988a205e1db84 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "d25ceaf478984a92871988a205e1db84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:19.378739 13610 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57542:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 08:00:19.385557 13864 tablet_service.cc:1467] Processing CreateTablet for tablet e4f7daf2c37a464ea1f1d0c31d61c780 (DEFAULT_TABLE table=rebalance_test_table_2 [id=15574b7b89a6476f81d68d4d6b612663]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:19.385620 13733 tablet_service.cc:1467] Processing CreateTablet for tablet 1bd357ad184842fa9c62fc038bc4ffc5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=15574b7b89a6476f81d68d4d6b612663]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:19.385572 13863 tablet_service.cc:1467] Processing CreateTablet for tablet a3c00d505e24482da03ca209c0665231 (DEFAULT_TABLE table=rebalance_test_table_2 [id=15574b7b89a6476f81d68d4d6b612663]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:19.385923 13733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bd357ad184842fa9c62fc038bc4ffc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:19.385918 13864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4f7daf2c37a464ea1f1d0c31d61c780. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:19.386116 13863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c00d505e24482da03ca209c0665231. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:19.387918 14229 tablet_bootstrap.cc:492] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03: Bootstrap starting.
I20240501 08:00:19.388056 14222 tablet_bootstrap.cc:492] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84: Bootstrap starting.
I20240501 08:00:19.388936 14229 tablet_bootstrap.cc:654] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:19.388972 14222 tablet_bootstrap.cc:654] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:19.390153 14222 tablet_bootstrap.cc:492] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84: No bootstrap required, opened a new log
I20240501 08:00:19.390271 14222 ts_tablet_manager.cc:1367] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:19.390621 14229 tablet_bootstrap.cc:492] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03: No bootstrap required, opened a new log
I20240501 08:00:19.390681 14222 raft_consensus.cc:348] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.390749 14229 ts_tablet_manager.cc:1367] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:19.390794 14222 raft_consensus.cc:374] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:19.390841 14222 raft_consensus.cc:729] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25ceaf478984a92871988a205e1db84, State: Initialized, Role: FOLLOWER
I20240501 08:00:19.390977 14222 consensus_queue.cc:260] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.391060 14222 raft_consensus.cc:388] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:19.391089 14222 raft_consensus.cc:482] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:19.391124 14222 raft_consensus.cc:3037] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:19.391481 14229 raft_consensus.cc:348] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.391604 14229 raft_consensus.cc:374] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:19.391650 14229 raft_consensus.cc:729] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c3e6b8e0ca438895a09b8f6aee5e03, State: Initialized, Role: FOLLOWER
I20240501 08:00:19.391808 14229 consensus_queue.cc:260] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.391902 14229 raft_consensus.cc:388] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:19.391968 14229 raft_consensus.cc:482] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:19.392007 14229 raft_consensus.cc:3037] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:19.392027 14222 raft_consensus.cc:504] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.392164 14222 leader_election.cc:302] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d25ceaf478984a92871988a205e1db84; no voters:
I20240501 08:00:19.392299 14222 leader_election.cc:288] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:19.392342 14226 raft_consensus.cc:2781] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:19.392411 14226 raft_consensus.cc:686] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [term 1 LEADER]: Becoming Leader. State: Replica: d25ceaf478984a92871988a205e1db84, State: Running, Role: LEADER
I20240501 08:00:19.392522 14226 consensus_queue.cc:237] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.392633 14222 ts_tablet_manager.cc:1398] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20240501 08:00:19.393059 14229 raft_consensus.cc:504] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.393308 14229 leader_election.cc:302] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10c3e6b8e0ca438895a09b8f6aee5e03; no voters:
I20240501 08:00:19.393417 14229 leader_election.cc:288] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:19.393479 14233 raft_consensus.cc:2781] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:19.393577 14233 raft_consensus.cc:686] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 LEADER]: Becoming Leader. State: Replica: 10c3e6b8e0ca438895a09b8f6aee5e03, State: Running, Role: LEADER
I20240501 08:00:19.393693 14233 consensus_queue.cc:237] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.393980 14229 ts_tablet_manager.cc:1398] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20240501 08:00:19.393999 13610 catalog_manager.cc:5484] T 1bd357ad184842fa9c62fc038bc4ffc5 P d25ceaf478984a92871988a205e1db84 reported cstate change: term changed from 0 to 1, leader changed from <none> to d25ceaf478984a92871988a205e1db84 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "d25ceaf478984a92871988a205e1db84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:19.394755 14229 tablet_bootstrap.cc:492] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03: Bootstrap starting.
I20240501 08:00:19.395350 13611 catalog_manager.cc:5484] T e4f7daf2c37a464ea1f1d0c31d61c780 P 10c3e6b8e0ca438895a09b8f6aee5e03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10c3e6b8e0ca438895a09b8f6aee5e03 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:19.398154 14229 tablet_bootstrap.cc:654] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:19.399721 14229 tablet_bootstrap.cc:492] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03: No bootstrap required, opened a new log
I20240501 08:00:19.399813 14229 ts_tablet_manager.cc:1367] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240501 08:00:19.400353 14229 raft_consensus.cc:348] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.400558 14229 raft_consensus.cc:374] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:19.400602 14229 raft_consensus.cc:729] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c3e6b8e0ca438895a09b8f6aee5e03, State: Initialized, Role: FOLLOWER
I20240501 08:00:19.400743 14229 consensus_queue.cc:260] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.400851 14229 raft_consensus.cc:388] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:19.400879 14229 raft_consensus.cc:482] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:19.400913 14229 raft_consensus.cc:3037] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:19.401618 14229 raft_consensus.cc:504] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.401743 14229 leader_election.cc:302] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10c3e6b8e0ca438895a09b8f6aee5e03; no voters:
I20240501 08:00:19.401854 14229 leader_election.cc:288] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:19.401949 14232 raft_consensus.cc:2781] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:19.402063 14232 raft_consensus.cc:686] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 LEADER]: Becoming Leader. State: Replica: 10c3e6b8e0ca438895a09b8f6aee5e03, State: Running, Role: LEADER
I20240501 08:00:19.402123 14229 ts_tablet_manager.cc:1398] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20240501 08:00:19.402191 14232 consensus_queue.cc:237] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.403779 13611 catalog_manager.cc:5484] T a3c00d505e24482da03ca209c0665231 P 10c3e6b8e0ca438895a09b8f6aee5e03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10c3e6b8e0ca438895a09b8f6aee5e03 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:19.581662 13611 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57542:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 08:00:19.587301 13732 tablet_service.cc:1467] Processing CreateTablet for tablet c0516c9b66cc4d45bc5f241ebf01f829 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ec285f63f5a542f58e75a10174b3d0c1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:19.587327 13863 tablet_service.cc:1467] Processing CreateTablet for tablet 8b557a1a792043409355be7dd0b4649c (DEFAULT_TABLE table=rebalance_test_table_3 [id=ec285f63f5a542f58e75a10174b3d0c1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:19.587658 13732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0516c9b66cc4d45bc5f241ebf01f829. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:19.587656 13863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b557a1a792043409355be7dd0b4649c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:19.587280 13733 tablet_service.cc:1467] Processing CreateTablet for tablet 979fe3f9c12440678e7700d51ce4f09c (DEFAULT_TABLE table=rebalance_test_table_3 [id=ec285f63f5a542f58e75a10174b3d0c1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:19.588008 13733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 979fe3f9c12440678e7700d51ce4f09c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:19.589408 14229 tablet_bootstrap.cc:492] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03: Bootstrap starting.
I20240501 08:00:19.589800 14222 tablet_bootstrap.cc:492] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84: Bootstrap starting.
I20240501 08:00:19.590556 14229 tablet_bootstrap.cc:654] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:19.590670 14222 tablet_bootstrap.cc:654] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:19.591675 14222 tablet_bootstrap.cc:492] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84: No bootstrap required, opened a new log
I20240501 08:00:19.591759 14222 ts_tablet_manager.cc:1367] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 08:00:19.591778 14229 tablet_bootstrap.cc:492] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03: No bootstrap required, opened a new log
I20240501 08:00:19.591851 14229 ts_tablet_manager.cc:1367] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 08:00:19.592304 14229 raft_consensus.cc:348] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.592299 14222 raft_consensus.cc:348] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.592401 14229 raft_consensus.cc:374] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:19.592401 14222 raft_consensus.cc:374] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:19.592423 14229 raft_consensus.cc:729] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c3e6b8e0ca438895a09b8f6aee5e03, State: Initialized, Role: FOLLOWER
I20240501 08:00:19.592432 14222 raft_consensus.cc:729] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25ceaf478984a92871988a205e1db84, State: Initialized, Role: FOLLOWER
I20240501 08:00:19.592547 14229 consensus_queue.cc:260] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.592559 14222 consensus_queue.cc:260] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.592617 14229 raft_consensus.cc:388] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:19.592630 14222 raft_consensus.cc:388] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:19.592640 14229 raft_consensus.cc:482] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:19.592670 14222 raft_consensus.cc:482] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:19.592677 14229 raft_consensus.cc:3037] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:19.592720 14222 raft_consensus.cc:3037] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:19.593425 14222 raft_consensus.cc:504] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.593418 14229 raft_consensus.cc:504] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.593554 14222 leader_election.cc:302] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d25ceaf478984a92871988a205e1db84; no voters:
I20240501 08:00:19.593554 14229 leader_election.cc:302] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10c3e6b8e0ca438895a09b8f6aee5e03; no voters:
I20240501 08:00:19.593636 14222 leader_election.cc:288] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:19.593663 14229 leader_election.cc:288] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:19.593717 14233 raft_consensus.cc:2781] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:19.593741 14226 raft_consensus.cc:2781] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:19.593791 14233 raft_consensus.cc:686] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [term 1 LEADER]: Becoming Leader. State: Replica: 10c3e6b8e0ca438895a09b8f6aee5e03, State: Running, Role: LEADER
I20240501 08:00:19.593809 14226 raft_consensus.cc:686] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [term 1 LEADER]: Becoming Leader. State: Replica: d25ceaf478984a92871988a205e1db84, State: Running, Role: LEADER
I20240501 08:00:19.593873 14222 ts_tablet_manager.cc:1398] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:19.593901 14233 consensus_queue.cc:237] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } }
I20240501 08:00:19.593933 14226 consensus_queue.cc:237] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.594031 14222 tablet_bootstrap.cc:492] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84: Bootstrap starting.
I20240501 08:00:19.594141 14229 ts_tablet_manager.cc:1398] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:19.595005 14222 tablet_bootstrap.cc:654] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:19.595434 13611 catalog_manager.cc:5484] T 8b557a1a792043409355be7dd0b4649c P 10c3e6b8e0ca438895a09b8f6aee5e03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10c3e6b8e0ca438895a09b8f6aee5e03 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c3e6b8e0ca438895a09b8f6aee5e03" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33271 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:19.596346 13610 catalog_manager.cc:5484] T 979fe3f9c12440678e7700d51ce4f09c P d25ceaf478984a92871988a205e1db84 reported cstate change: term changed from 0 to 1, leader changed from <none> to d25ceaf478984a92871988a205e1db84 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "d25ceaf478984a92871988a205e1db84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:19.596697 14222 tablet_bootstrap.cc:492] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84: No bootstrap required, opened a new log
I20240501 08:00:19.596802 14222 ts_tablet_manager.cc:1367] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 08:00:19.597244 14222 raft_consensus.cc:348] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.597342 14222 raft_consensus.cc:374] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:19.597376 14222 raft_consensus.cc:729] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25ceaf478984a92871988a205e1db84, State: Initialized, Role: FOLLOWER
I20240501 08:00:19.597492 14222 consensus_queue.cc:260] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.597679 14222 raft_consensus.cc:388] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:19.597800 14222 raft_consensus.cc:482] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:19.597856 14222 raft_consensus.cc:3037] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:19.599007 14222 raft_consensus.cc:504] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.599154 14222 leader_election.cc:302] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d25ceaf478984a92871988a205e1db84; no voters:
I20240501 08:00:19.599292 14222 leader_election.cc:288] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:19.599360 14225 raft_consensus.cc:2781] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:19.599452 14225 raft_consensus.cc:686] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [term 1 LEADER]: Becoming Leader. State: Replica: d25ceaf478984a92871988a205e1db84, State: Running, Role: LEADER
I20240501 08:00:19.599557 14225 consensus_queue.cc:237] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } }
I20240501 08:00:19.600232 14222 ts_tablet_manager.cc:1398] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20240501 08:00:19.601640 13611 catalog_manager.cc:5484] T c0516c9b66cc4d45bc5f241ebf01f829 P d25ceaf478984a92871988a205e1db84 reported cstate change: term changed from 0 to 1, leader changed from <none> to d25ceaf478984a92871988a205e1db84 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "d25ceaf478984a92871988a205e1db84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25ceaf478984a92871988a205e1db84" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37611 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:19.776981 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:37069
--local_ip_for_outbound_sockets=127.0.99.131
--tserver_master_addrs=127.0.99.190:42829
--webserver_port=34113
--webserver_interface=127.0.99.131
--builtin_ntp_servers=127.0.99.148:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 08:00:19.889531 14329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:19.889781 14329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:19.889829 14329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:19.893088 14329 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:19.893169 14329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:19.893255 14329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240501 08:00:19.897217 14329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:37069
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=34113
--tserver_master_addrs=127.0.99.190:42829
--never_fsync=true
--heap_profile_path=/tmp/kudu.14329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:19.898167 14329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:19.899268 14329 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:19.908453 14337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:19.908942 14335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:19.909502 14334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:19.910100 14329 server_base.cc:1034] running on GCE node
I20240501 08:00:19.910527 14329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:19.911137 14329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:19.912318 14329 hybrid_clock.cc:648] HybridClock initialized: now 1714550419912256 us; error 78 us; skew 500 ppm
I20240501 08:00:19.914316 14329 webserver.cc:460] Webserver started at http://127.0.99.131:34113/ using document root <none> and password file <none>
I20240501 08:00:19.914885 14329 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:19.914960 14329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:19.918390 14329 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:19.920428 14343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:19.921586 14329 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:19.921726 14329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "311f3bbbafd4430994bf923051c9f78a"
format_stamp: "Formatted at 2024-05-01 08:00:14 on dist-test-slave-f81v"
I20240501 08:00:19.922154 14329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:19.942142 14329 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:19.942813 14329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:19.942973 14329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:19.943531 14329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:19.944444 14329 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:19.944492 14329 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:19.944548 14329 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:19.944574 14329 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:19.953193 14329 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:37069
I20240501 08:00:19.953240 14456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:37069 every 8 connection(s)
I20240501 08:00:19.954244 14329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 08:00:19.963142 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 14329
I20240501 08:00:19.963541 14457 heartbeater.cc:347] Connected to a master server at 127.0.99.190:42829
I20240501 08:00:19.963670 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:45141
--local_ip_for_outbound_sockets=127.0.99.132
--tserver_master_addrs=127.0.99.190:42829
--webserver_port=45077
--webserver_interface=127.0.99.132
--builtin_ntp_servers=127.0.99.148:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:19.963819 14457 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:19.964507 14457 heartbeater.cc:511] Master 127.0.99.190:42829 requested a full tablet report, sending...
I20240501 08:00:19.965930 13611 ts_manager.cc:194] Re-registered known tserver with Master: 311f3bbbafd4430994bf923051c9f78a (127.0.99.131:37069)
I20240501 08:00:19.966500 13611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:40921
W20240501 08:00:20.084488 14460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:20.084745 14460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:20.084794 14460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:20.088027 14460 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:20.088127 14460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:20.088199 14460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240501 08:00:20.092496 14460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:45141
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=45077
--tserver_master_addrs=127.0.99.190:42829
--never_fsync=true
--heap_profile_path=/tmp/kudu.14460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:20.093503 14460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:20.094754 14460 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:20.104202 14466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:20.104585 14468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:20.105219 14465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:20.105731 14460 server_base.cc:1034] running on GCE node
I20240501 08:00:20.106153 14460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:20.106791 14460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:20.107993 14460 hybrid_clock.cc:648] HybridClock initialized: now 1714550420107974 us; error 31 us; skew 500 ppm
I20240501 08:00:20.110069 14460 webserver.cc:460] Webserver started at http://127.0.99.132:45077/ using document root <none> and password file <none>
I20240501 08:00:20.110633 14460 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:20.110709 14460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:20.114187 14460 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 08:00:20.116245 14474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:20.117348 14460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240501 08:00:20.117471 14460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "d44a3d2090f9401f91781f9404d58f04"
format_stamp: "Formatted at 2024-05-01 08:00:15 on dist-test-slave-f81v"
I20240501 08:00:20.117890 14460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:20.132503 14460 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:20.133143 14460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:20.133306 14460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:20.133839 14460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:20.134707 14460 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:20.134754 14460 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:20.134807 14460 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:20.134835 14460 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:20.142946 14460 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:45141
I20240501 08:00:20.142984 14587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:45141 every 8 connection(s)
I20240501 08:00:20.144004 14460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 08:00:20.150589 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 14460
I20240501 08:00:20.153316 14588 heartbeater.cc:347] Connected to a master server at 127.0.99.190:42829
I20240501 08:00:20.153551 14588 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:20.154181 14588 heartbeater.cc:511] Master 127.0.99.190:42829 requested a full tablet report, sending...
I20240501 08:00:20.155216 13611 ts_manager.cc:194] Re-registered known tserver with Master: d44a3d2090f9401f91781f9404d58f04 (127.0.99.132:45141)
I20240501 08:00:20.155813 13611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:59871
I20240501 08:00:20.348860 13863 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:20.349493 13733 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 08:00:20.350724 14391 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:20.351284 14522 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:20.479807 13863 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:20.481042 14522 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:20.481323 14391 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:20.481580 13733 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 08:00:20.586126 14522 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:20.586573 14391 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:20.586813 13733 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 08:00:20.586833 13863 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
9aea89d8f45c4585aeb7849d0b1c863e | 127.0.99.190:42829 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:41049 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
10c3e6b8e0ca438895a09b8f6aee5e03 | 127.0.99.130:33271 | HEALTHY | <none> | 6 | 0
311f3bbbafd4430994bf923051c9f78a | 127.0.99.131:37069 | HEALTHY | <none> | 0 | 0
d25ceaf478984a92871988a205e1db84 | 127.0.99.129:37611 | HEALTHY | <none> | 7 | 0
d44a3d2090f9401f91781f9404d58f04 | 127.0.99.132:45141 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 4 server(s) checked
local_ip_for_outbound_sockets | 127.0.99.129 | experimental | 127.0.99.129:37611
local_ip_for_outbound_sockets | 127.0.99.130 | experimental | 127.0.99.130:33271
local_ip_for_outbound_sockets | 127.0.99.131 | experimental | 127.0.99.131:37069
local_ip_for_outbound_sockets | 127.0.99.132 | experimental | 127.0.99.132:45141
never_fsync | true | unsafe,advanced | all 4 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 4 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 4 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
server_dump_info_format | pb | hidden | all 4 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.0.99.129:37611
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.0.99.130:33271
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.0.99.131:37069
server_dump_info_path | /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.0.99.132:45141
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:41049 | all 4 server(s) checked
time_source | builtin | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 6
Third Quartile | 7
Maximum | 7
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 5
Tablets | 13
Replicas | 13
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240501 08:00:20.616374 398 log_verifier.cc:126] Checking tablet 0ebf15eda0fa406eb2bf307a83c28f37
I20240501 08:00:20.624138 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 0ebf15eda0fa406eb2bf307a83c28f37
I20240501 08:00:20.624199 398 log_verifier.cc:126] Checking tablet 15b80f759a8e4e3783257b7e66667a5a
I20240501 08:00:20.631230 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 15b80f759a8e4e3783257b7e66667a5a
I20240501 08:00:20.631297 398 log_verifier.cc:126] Checking tablet 1bd357ad184842fa9c62fc038bc4ffc5
I20240501 08:00:20.638206 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 1bd357ad184842fa9c62fc038bc4ffc5
I20240501 08:00:20.638278 398 log_verifier.cc:126] Checking tablet 3d8ae6217d9b450cb07b930620fc5af0
I20240501 08:00:20.646438 398 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 3d8ae6217d9b450cb07b930620fc5af0
I20240501 08:00:20.646509 398 log_verifier.cc:126] Checking tablet 6404cb72b6b0473c837a1383cecaa197
I20240501 08:00:20.653748 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 6404cb72b6b0473c837a1383cecaa197
I20240501 08:00:20.653811 398 log_verifier.cc:126] Checking tablet 7ab30dbad3fd4dca9273431161444df1
I20240501 08:00:20.661865 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 7ab30dbad3fd4dca9273431161444df1
I20240501 08:00:20.661939 398 log_verifier.cc:126] Checking tablet 8b557a1a792043409355be7dd0b4649c
I20240501 08:00:20.669528 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 8b557a1a792043409355be7dd0b4649c
I20240501 08:00:20.669592 398 log_verifier.cc:126] Checking tablet 979fe3f9c12440678e7700d51ce4f09c
I20240501 08:00:20.676630 398 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 979fe3f9c12440678e7700d51ce4f09c
I20240501 08:00:20.676702 398 log_verifier.cc:126] Checking tablet a3c00d505e24482da03ca209c0665231
I20240501 08:00:20.683579 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet a3c00d505e24482da03ca209c0665231
I20240501 08:00:20.683655 398 log_verifier.cc:126] Checking tablet b5ed715f9be94deda847ab066a0d90bd
I20240501 08:00:20.691725 398 log_verifier.cc:177] Verified matching terms for 2 ops in tablet b5ed715f9be94deda847ab066a0d90bd
I20240501 08:00:20.691808 398 log_verifier.cc:126] Checking tablet c0516c9b66cc4d45bc5f241ebf01f829
I20240501 08:00:20.699056 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet c0516c9b66cc4d45bc5f241ebf01f829
I20240501 08:00:20.699131 398 log_verifier.cc:126] Checking tablet e4f7daf2c37a464ea1f1d0c31d61c780
I20240501 08:00:20.706589 398 log_verifier.cc:177] Verified matching terms for 2 ops in tablet e4f7daf2c37a464ea1f1d0c31d61c780
I20240501 08:00:20.706650 398 log_verifier.cc:126] Checking tablet ed20fa3af025445fbb10e0beac7c6da9
I20240501 08:00:20.713474 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet ed20fa3af025445fbb10e0beac7c6da9
I20240501 08:00:20.714129 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 13671
I20240501 08:00:20.719770 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 13802
I20240501 08:00:20.724957 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 14329
I20240501 08:00:20.730000 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 14460
I20240501 08:00:20.734675 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 13580
2024-05-01T08:00:20Z chronyd exiting
[ OK ] RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove/5 (6770 ms)
[----------] 1 test from RebalancerAndSingleReplicaTablets (6770 ms total)
[----------] 2 tests from LocationAwareRebalancingParamTest
[ RUN ] LocationAwareRebalancingParamTest.Rebalance/idx7_rf2_t3_l_1_2_1
I20240501 08:00:20.763857 398 test_util.cc:268] Using random seed: -523799349
I20240501 08:00:20.764662 398 ts_itest-base.cc:112] Starting cluster with:
I20240501 08:00:20.764717 398 ts_itest-base.cc:113] --------------
I20240501 08:00:20.764734 398 ts_itest-base.cc:114] 4 tablet servers
I20240501 08:00:20.764748 398 ts_itest-base.cc:115] 2 replicas per TS
I20240501 08:00:20.764760 398 ts_itest-base.cc:116] --------------
2024-05-01T08:00:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T08:00:20Z Disabled control of system clock
I20240501 08:00:20.772711 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:40571
--webserver_interface=127.0.99.190
--webserver_port=0
--builtin_ntp_servers=127.0.99.148:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.99.190:40571
--location_mapping_cmd=/tmp/dist-test-taskrYyCEf/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/location-assignment.state --map /A:1 --map /B:2 --map /C:1
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false
--master_client_location_assignment_enabled=false
--allow_unsafe_replication_factor with env {}
W20240501 08:00:20.878062 14697 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240501 08:00:20.878348 14697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:20.878374 14697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:20.878410 14697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:20.881664 14697 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:20.881772 14697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 08:00:20.881799 14697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:20.881819 14697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 08:00:20.881834 14697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 08:00:20.885818 14697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--location_mapping_cmd=/tmp/dist-test-taskrYyCEf/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/location-assignment.state --map /A:1 --map /B:2 --map /C:1
--ipki_ca_key_size=768
--master_addresses=127.0.99.190:40571
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:40571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.99.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.14697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:20.886730 14697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:20.887830 14697 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:20.895534 14702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:20.896011 14705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:20.896554 14703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:20.897207 14697 server_base.cc:1034] running on GCE node
I20240501 08:00:20.897765 14697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:20.898741 14697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:20.899902 14697 hybrid_clock.cc:648] HybridClock initialized: now 1714550420899886 us; error 39 us; skew 500 ppm
I20240501 08:00:20.901885 14697 webserver.cc:460] Webserver started at http://127.0.99.190:46119/ using document root <none> and password file <none>
I20240501 08:00:20.902432 14697 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:20.902493 14697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:20.902659 14697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:20.904357 14697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "584fa849a5e64433aeb549e40f6108be"
format_stamp: "Formatted at 2024-05-01 08:00:20 on dist-test-slave-f81v"
I20240501 08:00:20.904837 14697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "584fa849a5e64433aeb549e40f6108be"
format_stamp: "Formatted at 2024-05-01 08:00:20 on dist-test-slave-f81v"
I20240501 08:00:20.908270 14697 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:20.910568 14711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:20.911597 14697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:20.911728 14697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
uuid: "584fa849a5e64433aeb549e40f6108be"
format_stamp: "Formatted at 2024-05-01 08:00:20 on dist-test-slave-f81v"
I20240501 08:00:20.911823 14697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:20.925148 14697 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:20.925861 14697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:20.926025 14697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:20.932436 14697 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:40571
I20240501 08:00:20.932449 14763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:40571 every 8 connection(s)
I20240501 08:00:20.933406 14697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 08:00:20.936380 14764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:20.938024 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 14697
I20240501 08:00:20.938176 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 08:00:20.942404 14764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be: Bootstrap starting.
I20240501 08:00:20.944561 14764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:20.945305 14764 log.cc:824] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:20.946955 14764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be: No bootstrap required, opened a new log
I20240501 08:00:20.949476 14764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584fa849a5e64433aeb549e40f6108be" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 40571 } }
I20240501 08:00:20.949672 14764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:20.949733 14764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 584fa849a5e64433aeb549e40f6108be, State: Initialized, Role: FOLLOWER
I20240501 08:00:20.950182 14764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584fa849a5e64433aeb549e40f6108be" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 40571 } }
I20240501 08:00:20.950307 14764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:20.950353 14764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:20.950430 14764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:20.951145 14764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584fa849a5e64433aeb549e40f6108be" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 40571 } }
I20240501 08:00:20.951438 14764 leader_election.cc:302] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 584fa849a5e64433aeb549e40f6108be; no voters:
I20240501 08:00:20.951695 14764 leader_election.cc:288] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:20.951795 14769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:20.951980 14769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [term 1 LEADER]: Becoming Leader. State: Replica: 584fa849a5e64433aeb549e40f6108be, State: Running, Role: LEADER
I20240501 08:00:20.952649 14769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584fa849a5e64433aeb549e40f6108be" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 40571 } }
I20240501 08:00:20.952909 14764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [sys.catalog]: configured and running, proceeding with master startup.
I20240501 08:00:20.954257 14770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "584fa849a5e64433aeb549e40f6108be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584fa849a5e64433aeb549e40f6108be" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 40571 } } }
I20240501 08:00:20.954257 14771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 584fa849a5e64433aeb549e40f6108be. Latest consensus state: current_term: 1 leader_uuid: "584fa849a5e64433aeb549e40f6108be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584fa849a5e64433aeb549e40f6108be" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 40571 } } }
I20240501 08:00:20.954391 14770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [sys.catalog]: This master's current role is: LEADER
I20240501 08:00:20.954397 14771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be [sys.catalog]: This master's current role is: LEADER
I20240501 08:00:20.954921 14780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 08:00:20.957780 14780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 08:00:20.962814 14780 catalog_manager.cc:1347] Generated new cluster ID: b770b105e6014ee581c3dea943f2e3a0
I20240501 08:00:20.962886 14780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 08:00:20.979585 14780 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 08:00:20.980965 14780 catalog_manager.cc:1504] Loading token signing keys...
I20240501 08:00:20.989025 14780 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 584fa849a5e64433aeb549e40f6108be: Generated new TSK 0
I20240501 08:00:20.990118 14780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 08:00:21.004518 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.129:0
--local_ip_for_outbound_sockets=127.0.99.129
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:40571
--builtin_ntp_servers=127.0.99.148:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 08:00:21.120923 14788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:21.121184 14788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:21.121234 14788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:21.124455 14788 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:21.124531 14788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:21.124603 14788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.129
I20240501 08:00:21.128798 14788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:40571
--never_fsync=true
--heap_profile_path=/tmp/kudu.14788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.129
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:21.130164 14788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:21.131464 14788 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:21.140966 14796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:21.141920 14793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:21.142189 14794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:21.142843 14788 server_base.cc:1034] running on GCE node
I20240501 08:00:21.143316 14788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:21.144217 14788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:21.145377 14788 hybrid_clock.cc:648] HybridClock initialized: now 1714550421145352 us; error 44 us; skew 500 ppm
I20240501 08:00:21.151614 14788 webserver.cc:460] Webserver started at http://127.0.99.129:38807/ using document root <none> and password file <none>
I20240501 08:00:21.152290 14788 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:21.152354 14788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:21.152541 14788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:21.154788 14788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "6016871632a84df2abdcc5690f07cb9e"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:21.155313 14788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "6016871632a84df2abdcc5690f07cb9e"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:21.158970 14788 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20240501 08:00:21.161366 14802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:21.162559 14788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:21.162674 14788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "6016871632a84df2abdcc5690f07cb9e"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:21.162779 14788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:21.172395 14788 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:21.173009 14788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:21.173174 14788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:21.173755 14788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:21.174655 14788 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:21.174700 14788 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:21.174746 14788 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:21.174772 14788 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:21.184054 14788 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:39697
I20240501 08:00:21.184078 14915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:39697 every 8 connection(s)
I20240501 08:00:21.185055 14788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 08:00:21.191238 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 14788
I20240501 08:00:21.191395 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 08:00:21.194455 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:0
--local_ip_for_outbound_sockets=127.0.99.130
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:40571
--builtin_ntp_servers=127.0.99.148:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:21.196455 14916 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40571
I20240501 08:00:21.196743 14916 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:21.197440 14916 heartbeater.cc:511] Master 127.0.99.190:40571 requested a full tablet report, sending...
I20240501 08:00:21.233006 14728 ts_manager.cc:194] Registered new tserver with Master: 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:21.234762 14728 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.129:33721
W20240501 08:00:21.304078 14919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:21.304316 14919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:21.304360 14919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:21.307379 14919 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:21.307451 14919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:21.307518 14919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240501 08:00:21.311180 14919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:40571
--never_fsync=true
--heap_profile_path=/tmp/kudu.14919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:21.312146 14919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:21.313217 14919 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:21.321765 14925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:21.322258 14928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:21.322755 14926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:21.323330 14919 server_base.cc:1034] running on GCE node
I20240501 08:00:21.323714 14919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:21.324388 14919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:21.325564 14919 hybrid_clock.cc:648] HybridClock initialized: now 1714550421325546 us; error 33 us; skew 500 ppm
I20240501 08:00:21.327471 14919 webserver.cc:460] Webserver started at http://127.0.99.130:40725/ using document root <none> and password file <none>
I20240501 08:00:21.328056 14919 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:21.328119 14919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:21.328299 14919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:21.330092 14919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "58593562dc504df98fbba7f0b734958f"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:21.330562 14919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "58593562dc504df98fbba7f0b734958f"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:21.333950 14919 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 08:00:21.336624 14934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:21.337785 14919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:21.337908 14919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "58593562dc504df98fbba7f0b734958f"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:21.338008 14919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:21.363207 14919 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:21.363869 14919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:21.364064 14919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:21.364609 14919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:21.365556 14919 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:21.365607 14919 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:21.365648 14919 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:21.365660 14919 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:21.373958 14919 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:41275
I20240501 08:00:21.373996 15047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:41275 every 8 connection(s)
I20240501 08:00:21.374876 14919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 08:00:21.381165 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 14919
I20240501 08:00:21.381279 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 08:00:21.383445 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:0
--local_ip_for_outbound_sockets=127.0.99.131
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:40571
--builtin_ntp_servers=127.0.99.148:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:21.384004 15048 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40571
I20240501 08:00:21.384253 15048 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:21.384897 15048 heartbeater.cc:511] Master 127.0.99.190:40571 requested a full tablet report, sending...
I20240501 08:00:21.418118 14728 ts_manager.cc:194] Registered new tserver with Master: 58593562dc504df98fbba7f0b734958f (127.0.99.130:41275)
I20240501 08:00:21.418817 14728 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:47303
W20240501 08:00:21.493700 15051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:21.493938 15051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:21.494000 15051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:21.497107 15051 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:21.497193 15051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:21.497282 15051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240501 08:00:21.501039 15051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:40571
--never_fsync=true
--heap_profile_path=/tmp/kudu.15051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:21.501961 15051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:21.503072 15051 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:21.512347 15058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:21.512806 15057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:21.513267 15060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:21.513932 15051 server_base.cc:1034] running on GCE node
I20240501 08:00:21.514387 15051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:21.515056 15051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:21.516228 15051 hybrid_clock.cc:648] HybridClock initialized: now 1714550421516206 us; error 37 us; skew 500 ppm
I20240501 08:00:21.518007 15051 webserver.cc:460] Webserver started at http://127.0.99.131:45361/ using document root <none> and password file <none>
I20240501 08:00:21.518559 15051 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:21.518617 15051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:21.518783 15051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:21.520464 15051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:21.520905 15051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:21.524210 15051 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 08:00:21.526400 15066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:21.527407 15051 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:21.527518 15051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:21.527607 15051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:21.569851 15051 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:21.570470 15051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:21.570637 15051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:21.571189 15051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:21.572110 15051 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:21.572158 15051 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:21.572211 15051 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:21.572237 15051 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:21.580912 15051 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:34069
I20240501 08:00:21.580926 15179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:34069 every 8 connection(s)
I20240501 08:00:21.581880 15051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 08:00:21.590186 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 15051
I20240501 08:00:21.590308 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 08:00:21.590812 15180 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40571
I20240501 08:00:21.591048 15180 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:21.591595 15180 heartbeater.cc:511] Master 127.0.99.190:40571 requested a full tablet report, sending...
I20240501 08:00:21.592367 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:0
--local_ip_for_outbound_sockets=127.0.99.132
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:40571
--builtin_ntp_servers=127.0.99.148:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:21.625797 14728 ts_manager.cc:194] Registered new tserver with Master: 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131:34069)
I20240501 08:00:21.626708 14728 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:43313
W20240501 08:00:21.705369 15183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:21.705618 15183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:21.705662 15183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:21.708935 15183 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:21.709005 15183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:21.709074 15183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240501 08:00:21.712797 15183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:40571
--never_fsync=true
--heap_profile_path=/tmp/kudu.15183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:21.713696 15183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:21.714751 15183 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:21.723559 15189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:21.724102 15192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:21.724686 15190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:21.725172 15183 server_base.cc:1034] running on GCE node
I20240501 08:00:21.725595 15183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:21.726295 15183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:21.727459 15183 hybrid_clock.cc:648] HybridClock initialized: now 1714550421727440 us; error 35 us; skew 500 ppm
I20240501 08:00:21.729573 15183 webserver.cc:460] Webserver started at http://127.0.99.132:35619/ using document root <none> and password file <none>
I20240501 08:00:21.730214 15183 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:21.730276 15183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:21.730475 15183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:21.732290 15183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "e019f42a04ca40b491694f86026e3638"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:21.732767 15183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "e019f42a04ca40b491694f86026e3638"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:21.736266 15183 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240501 08:00:21.738622 15198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:21.739706 15183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:21.739821 15183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "e019f42a04ca40b491694f86026e3638"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:21.739907 15183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:21.753134 15183 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:21.753721 15183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:21.753876 15183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:21.754421 15183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:21.755295 15183 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:21.755338 15183 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:21.755383 15183 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:21.755402 15183 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:21.763286 15183 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:40573
I20240501 08:00:21.763309 15311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:40573 every 8 connection(s)
I20240501 08:00:21.764294 15183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 08:00:21.769543 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 15183
I20240501 08:00:21.769675 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 08:00:21.773847 15312 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40571
I20240501 08:00:21.774107 15312 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:21.774670 15312 heartbeater.cc:511] Master 127.0.99.190:40571 requested a full tablet report, sending...
I20240501 08:00:21.808451 14728 ts_manager.cc:194] Registered new tserver with Master: e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573)
I20240501 08:00:21.809147 14728 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:56569
I20240501 08:00:21.820089 398 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240501 08:00:21.829061 398 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240501 08:00:21.829142 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 14919
I20240501 08:00:21.833979 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 15051
I20240501 08:00:22.238159 14916 heartbeater.cc:503] Master 127.0.99.190:40571 was elected leader, sending a full tablet report...
I20240501 08:00:22.811285 15312 heartbeater.cc:503] Master 127.0.99.190:40571 was elected leader, sending a full tablet report...
I20240501 08:00:25.590231 14728 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34210:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 08:00:25.593667 14728 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240501 08:00:25.609645 14850 tablet_service.cc:1467] Processing CreateTablet for tablet fe884e808db647378c273f2ac6988b32 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6f7b30c496af4f26b0be4a9f8e72ada4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:25.609766 15245 tablet_service.cc:1467] Processing CreateTablet for tablet 3ed99c9a3b7542eca1199ff105198318 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6f7b30c496af4f26b0be4a9f8e72ada4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:25.610129 14849 tablet_service.cc:1467] Processing CreateTablet for tablet 3ed99c9a3b7542eca1199ff105198318 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6f7b30c496af4f26b0be4a9f8e72ada4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:25.609645 15246 tablet_service.cc:1467] Processing CreateTablet for tablet fe884e808db647378c273f2ac6988b32 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6f7b30c496af4f26b0be4a9f8e72ada4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:25.610643 15245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ed99c9a3b7542eca1199ff105198318. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:25.610641 14849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ed99c9a3b7542eca1199ff105198318. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:25.609647 14848 tablet_service.cc:1467] Processing CreateTablet for tablet eb631e0b15744ca4b5f767f06711a3a7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6f7b30c496af4f26b0be4a9f8e72ada4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:25.609647 15244 tablet_service.cc:1467] Processing CreateTablet for tablet eb631e0b15744ca4b5f767f06711a3a7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6f7b30c496af4f26b0be4a9f8e72ada4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:25.610980 15244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb631e0b15744ca4b5f767f06711a3a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:25.611085 14848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb631e0b15744ca4b5f767f06711a3a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:25.611865 15246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe884e808db647378c273f2ac6988b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:25.615662 14850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe884e808db647378c273f2ac6988b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:25.618108 15330 tablet_bootstrap.cc:492] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638: Bootstrap starting.
I20240501 08:00:25.618431 15329 tablet_bootstrap.cc:492] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e: Bootstrap starting.
I20240501 08:00:25.620108 15329 tablet_bootstrap.cc:654] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:25.620127 15330 tablet_bootstrap.cc:654] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:25.620824 15329 log.cc:824] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:25.621119 15330 log.cc:824] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:25.622875 15329 tablet_bootstrap.cc:492] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e: No bootstrap required, opened a new log
I20240501 08:00:25.623085 15329 ts_tablet_manager.cc:1367] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20240501 08:00:25.623175 15330 tablet_bootstrap.cc:492] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638: No bootstrap required, opened a new log
I20240501 08:00:25.623337 15330 ts_tablet_manager.cc:1367] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20240501 08:00:25.626471 15330 raft_consensus.cc:348] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:25.626468 15329 raft_consensus.cc:348] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:25.626667 15330 raft_consensus.cc:374] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:25.626668 15329 raft_consensus.cc:374] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:25.626701 15330 raft_consensus.cc:729] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e019f42a04ca40b491694f86026e3638, State: Initialized, Role: FOLLOWER
I20240501 08:00:25.626713 15329 raft_consensus.cc:729] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Initialized, Role: FOLLOWER
I20240501 08:00:25.627120 15329 consensus_queue.cc:260] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:25.627166 15330 consensus_queue.cc:260] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:25.627987 15330 ts_tablet_manager.cc:1398] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20240501 08:00:25.628371 15330 tablet_bootstrap.cc:492] T fe884e808db647378c273f2ac6988b32 P e019f42a04ca40b491694f86026e3638: Bootstrap starting.
I20240501 08:00:25.629439 15330 tablet_bootstrap.cc:654] T fe884e808db647378c273f2ac6988b32 P e019f42a04ca40b491694f86026e3638: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:25.629266 15329 ts_tablet_manager.cc:1398] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20240501 08:00:25.630756 15330 tablet_bootstrap.cc:492] T fe884e808db647378c273f2ac6988b32 P e019f42a04ca40b491694f86026e3638: No bootstrap required, opened a new log
I20240501 08:00:25.630841 15330 ts_tablet_manager.cc:1367] T fe884e808db647378c273f2ac6988b32 P e019f42a04ca40b491694f86026e3638: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:25.631155 15329 tablet_bootstrap.cc:492] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e: Bootstrap starting.
I20240501 08:00:25.631572 15330 raft_consensus.cc:348] T fe884e808db647378c273f2ac6988b32 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.631686 15330 raft_consensus.cc:374] T fe884e808db647378c273f2ac6988b32 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:25.631718 15330 raft_consensus.cc:729] T fe884e808db647378c273f2ac6988b32 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e019f42a04ca40b491694f86026e3638, State: Initialized, Role: FOLLOWER
I20240501 08:00:25.631861 15330 consensus_queue.cc:260] T fe884e808db647378c273f2ac6988b32 P e019f42a04ca40b491694f86026e3638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.632189 15330 ts_tablet_manager.cc:1398] T fe884e808db647378c273f2ac6988b32 P e019f42a04ca40b491694f86026e3638: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:25.632337 15330 tablet_bootstrap.cc:492] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638: Bootstrap starting.
I20240501 08:00:25.633216 15330 tablet_bootstrap.cc:654] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:25.633409 15329 tablet_bootstrap.cc:654] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:25.634339 15330 tablet_bootstrap.cc:492] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638: No bootstrap required, opened a new log
I20240501 08:00:25.634408 15330 ts_tablet_manager.cc:1367] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:25.634444 15329 tablet_bootstrap.cc:492] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e: No bootstrap required, opened a new log
I20240501 08:00:25.634531 15329 ts_tablet_manager.cc:1367] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 08:00:25.634778 15330 raft_consensus.cc:348] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.634863 15330 raft_consensus.cc:374] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:25.634883 15330 raft_consensus.cc:729] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e019f42a04ca40b491694f86026e3638, State: Initialized, Role: FOLLOWER
I20240501 08:00:25.635004 15330 consensus_queue.cc:260] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.635069 15329 raft_consensus.cc:348] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.635177 15329 raft_consensus.cc:374] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:25.635205 15329 raft_consensus.cc:729] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Initialized, Role: FOLLOWER
I20240501 08:00:25.635233 15330 ts_tablet_manager.cc:1398] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:25.635320 15329 consensus_queue.cc:260] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.635586 15329 ts_tablet_manager.cc:1398] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:25.635746 15329 tablet_bootstrap.cc:492] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e: Bootstrap starting.
I20240501 08:00:25.636629 15329 tablet_bootstrap.cc:654] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:25.637952 15329 tablet_bootstrap.cc:492] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e: No bootstrap required, opened a new log
I20240501 08:00:25.638037 15329 ts_tablet_manager.cc:1367] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:25.638392 15329 raft_consensus.cc:348] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.638478 15329 raft_consensus.cc:374] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:25.638505 15329 raft_consensus.cc:729] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Initialized, Role: FOLLOWER
I20240501 08:00:25.638622 15329 consensus_queue.cc:260] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.638830 15329 ts_tablet_manager.cc:1398] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:25.659116 15333 raft_consensus.cc:482] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:25.659302 15333 raft_consensus.cc:504] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.660373 15333 leader_election.cc:288] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573)
I20240501 08:00:25.664412 15266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe884e808db647378c273f2ac6988b32" candidate_uuid: "6016871632a84df2abdcc5690f07cb9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e019f42a04ca40b491694f86026e3638" is_pre_election: true
I20240501 08:00:25.664752 15266 raft_consensus.cc:2454] T fe884e808db647378c273f2ac6988b32 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6016871632a84df2abdcc5690f07cb9e in term 0.
I20240501 08:00:25.665241 14806 leader_election.cc:302] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:25.665493 15333 raft_consensus.cc:2781] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:25.665562 15333 raft_consensus.cc:482] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:25.665591 15333 raft_consensus.cc:3037] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:25.668232 15333 raft_consensus.cc:504] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.668601 15333 leader_election.cc:288] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 election: Requested vote from peers e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573)
I20240501 08:00:25.669039 15266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe884e808db647378c273f2ac6988b32" candidate_uuid: "6016871632a84df2abdcc5690f07cb9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e019f42a04ca40b491694f86026e3638"
I20240501 08:00:25.669159 15266 raft_consensus.cc:3037] T fe884e808db647378c273f2ac6988b32 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:25.670351 15266 raft_consensus.cc:2454] T fe884e808db647378c273f2ac6988b32 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6016871632a84df2abdcc5690f07cb9e in term 1.
I20240501 08:00:25.670678 14806 leader_election.cc:302] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:25.670871 15333 raft_consensus.cc:2781] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:25.671090 15333 raft_consensus.cc:686] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Becoming Leader. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Running, Role: LEADER
I20240501 08:00:25.671474 15333 consensus_queue.cc:237] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.674481 14727 catalog_manager.cc:5484] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6016871632a84df2abdcc5690f07cb9e (127.0.99.129). New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: HEALTHY } } }
W20240501 08:00:25.689774 14917 tablet.cc:2369] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 08:00:25.730829 15334 raft_consensus.cc:482] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:25.730999 15334 raft_consensus.cc:504] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.732069 15334 leader_election.cc:288] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:25.736610 14870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb631e0b15744ca4b5f767f06711a3a7" candidate_uuid: "e019f42a04ca40b491694f86026e3638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6016871632a84df2abdcc5690f07cb9e" is_pre_election: true
I20240501 08:00:25.736855 14870 raft_consensus.cc:2454] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e019f42a04ca40b491694f86026e3638 in term 0.
I20240501 08:00:25.737355 15200 leader_election.cc:302] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:25.737666 15334 raft_consensus.cc:2781] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:25.737766 15334 raft_consensus.cc:482] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:25.737814 15334 raft_consensus.cc:3037] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:25.738756 15334 raft_consensus.cc:504] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.739086 15334 leader_election.cc:288] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 election: Requested vote from peers 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:25.739602 14870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb631e0b15744ca4b5f767f06711a3a7" candidate_uuid: "e019f42a04ca40b491694f86026e3638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6016871632a84df2abdcc5690f07cb9e"
I20240501 08:00:25.739732 14870 raft_consensus.cc:3037] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:25.741283 14870 raft_consensus.cc:2454] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e019f42a04ca40b491694f86026e3638 in term 1.
I20240501 08:00:25.741886 15200 leader_election.cc:302] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:25.742199 15334 raft_consensus.cc:2781] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:25.742476 15334 raft_consensus.cc:686] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Becoming Leader. State: Replica: e019f42a04ca40b491694f86026e3638, State: Running, Role: LEADER
I20240501 08:00:25.742805 15334 consensus_queue.cc:237] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:25.747107 14727 catalog_manager.cc:5484] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 reported cstate change: term changed from 0 to 1, leader changed from <none> to e019f42a04ca40b491694f86026e3638 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: UNKNOWN } } }
W20240501 08:00:25.768414 15313 tablet.cc:2369] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 08:00:25.999043 15333 raft_consensus.cc:482] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:25.999218 15333 raft_consensus.cc:504] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:25.999614 15333 leader_election.cc:288] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573)
I20240501 08:00:26.000201 15266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ed99c9a3b7542eca1199ff105198318" candidate_uuid: "6016871632a84df2abdcc5690f07cb9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e019f42a04ca40b491694f86026e3638" is_pre_election: true
I20240501 08:00:26.000344 15266 raft_consensus.cc:2454] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6016871632a84df2abdcc5690f07cb9e in term 0.
I20240501 08:00:26.000659 14806 leader_election.cc:302] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:26.000833 15333 raft_consensus.cc:2781] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:26.000888 15333 raft_consensus.cc:482] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:26.000919 15333 raft_consensus.cc:3037] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:26.001801 15333 raft_consensus.cc:504] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.002070 15333 leader_election.cc:288] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 election: Requested vote from peers e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573)
I20240501 08:00:26.002496 15266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ed99c9a3b7542eca1199ff105198318" candidate_uuid: "6016871632a84df2abdcc5690f07cb9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e019f42a04ca40b491694f86026e3638"
I20240501 08:00:26.002621 15266 raft_consensus.cc:3037] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:26.003391 15266 raft_consensus.cc:2454] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6016871632a84df2abdcc5690f07cb9e in term 1.
I20240501 08:00:26.003737 14806 leader_election.cc:302] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:26.003965 15333 raft_consensus.cc:2781] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:26.004084 15333 raft_consensus.cc:686] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Becoming Leader. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Running, Role: LEADER
I20240501 08:00:26.004222 15333 consensus_queue.cc:237] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.005887 14727 catalog_manager.cc:5484] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6016871632a84df2abdcc5690f07cb9e (127.0.99.129). New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: UNKNOWN } } }
W20240501 08:00:26.186736 15346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:26.187032 15346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:26.190392 15346 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240501 08:00:26.221933 15333 consensus_queue.cc:1035] T fe884e808db647378c273f2ac6988b32 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:26.298709 15266 raft_consensus.cc:1261] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:26.299224 15340 consensus_queue.cc:1035] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:26.303182 14870 raft_consensus.cc:1261] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:26.304062 15334 consensus_queue.cc:1035] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:26.308815 15364 mvcc.cc:205] Tried to move back new op lower bound from 7022798546114035712 to 7022798544914149376. Current Snapshot: MvccSnapshot[applied={T|T < 7022798546114035712}]
I20240501 08:00:26.313130 15379 mvcc.cc:205] Tried to move back new op lower bound from 7022798546123386880 to 7022798543845965824. Current Snapshot: MvccSnapshot[applied={T|T < 7022798546123386880}]
INSERT report
rows total: 2000
time total: 97.5118 ms
time per row: 0.0487559 ms
I20240501 08:00:26.345058 14727 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34210:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 08:00:26.345618 14727 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_1 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240501 08:00:26.350677 14850 tablet_service.cc:1467] Processing CreateTablet for tablet 3ce0e31c02ff4701954a6c052fdae018 (DEFAULT_TABLE table=rebalance_test_table_1 [id=b74463b6b30c4a46989fc21f0cb1ec68]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:26.350677 14848 tablet_service.cc:1467] Processing CreateTablet for tablet aaebe02e13964ab88e1f6c24a9ce8131 (DEFAULT_TABLE table=rebalance_test_table_1 [id=b74463b6b30c4a46989fc21f0cb1ec68]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:26.351006 14848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaebe02e13964ab88e1f6c24a9ce8131. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:26.350975 15244 tablet_service.cc:1467] Processing CreateTablet for tablet 3ce0e31c02ff4701954a6c052fdae018 (DEFAULT_TABLE table=rebalance_test_table_1 [id=b74463b6b30c4a46989fc21f0cb1ec68]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:26.351174 14850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ce0e31c02ff4701954a6c052fdae018. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:26.351251 15244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ce0e31c02ff4701954a6c052fdae018. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:26.352226 15245 tablet_service.cc:1467] Processing CreateTablet for tablet aaebe02e13964ab88e1f6c24a9ce8131 (DEFAULT_TABLE table=rebalance_test_table_1 [id=b74463b6b30c4a46989fc21f0cb1ec68]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:26.352458 15245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaebe02e13964ab88e1f6c24a9ce8131. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:26.353461 15395 tablet_bootstrap.cc:492] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e: Bootstrap starting.
I20240501 08:00:26.354297 15395 tablet_bootstrap.cc:654] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:26.354676 14848 tablet_service.cc:1467] Processing CreateTablet for tablet f809a3f2420f4e8398cd05529886691b (DEFAULT_TABLE table=rebalance_test_table_1 [id=b74463b6b30c4a46989fc21f0cb1ec68]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:26.354943 14848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f809a3f2420f4e8398cd05529886691b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:26.354981 15394 tablet_bootstrap.cc:492] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638: Bootstrap starting.
I20240501 08:00:26.355067 15245 tablet_service.cc:1467] Processing CreateTablet for tablet f809a3f2420f4e8398cd05529886691b (DEFAULT_TABLE table=rebalance_test_table_1 [id=b74463b6b30c4a46989fc21f0cb1ec68]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:26.355284 15245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f809a3f2420f4e8398cd05529886691b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:26.355643 15395 tablet_bootstrap.cc:492] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e: No bootstrap required, opened a new log
I20240501 08:00:26.355768 15395 ts_tablet_manager.cc:1367] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:26.355803 15394 tablet_bootstrap.cc:654] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:26.356523 15395 raft_consensus.cc:348] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.356698 15395 raft_consensus.cc:374] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:26.356775 15395 raft_consensus.cc:729] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Initialized, Role: FOLLOWER
I20240501 08:00:26.356958 15395 consensus_queue.cc:260] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.357657 15395 ts_tablet_manager.cc:1398] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:26.357709 15394 tablet_bootstrap.cc:492] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638: No bootstrap required, opened a new log
I20240501 08:00:26.357798 15394 ts_tablet_manager.cc:1367] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 08:00:26.357895 15395 tablet_bootstrap.cc:492] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e: Bootstrap starting.
I20240501 08:00:26.358204 15394 raft_consensus.cc:348] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.358289 15394 raft_consensus.cc:374] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:26.358315 15394 raft_consensus.cc:729] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e019f42a04ca40b491694f86026e3638, State: Initialized, Role: FOLLOWER
I20240501 08:00:26.358461 15394 consensus_queue.cc:260] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.358719 15394 ts_tablet_manager.cc:1398] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:00:26.358901 15394 tablet_bootstrap.cc:492] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638: Bootstrap starting.
I20240501 08:00:26.359088 15395 tablet_bootstrap.cc:654] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:26.359722 15394 tablet_bootstrap.cc:654] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:26.360924 15395 tablet_bootstrap.cc:492] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e: No bootstrap required, opened a new log
I20240501 08:00:26.360999 15395 ts_tablet_manager.cc:1367] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:26.361228 15394 tablet_bootstrap.cc:492] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638: No bootstrap required, opened a new log
I20240501 08:00:26.361306 15394 ts_tablet_manager.cc:1367] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 08:00:26.361382 15395 raft_consensus.cc:348] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.361475 15395 raft_consensus.cc:374] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:26.361496 15395 raft_consensus.cc:729] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Initialized, Role: FOLLOWER
I20240501 08:00:26.361610 15395 consensus_queue.cc:260] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.361846 15395 ts_tablet_manager.cc:1398] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:26.361855 15394 raft_consensus.cc:348] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.361958 15394 raft_consensus.cc:374] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:26.361995 15394 raft_consensus.cc:729] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e019f42a04ca40b491694f86026e3638, State: Initialized, Role: FOLLOWER
I20240501 08:00:26.362005 15395 tablet_bootstrap.cc:492] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e: Bootstrap starting.
I20240501 08:00:26.362110 15394 consensus_queue.cc:260] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.362715 15394 ts_tablet_manager.cc:1398] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:00:26.362879 15394 tablet_bootstrap.cc:492] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638: Bootstrap starting.
I20240501 08:00:26.363044 15395 tablet_bootstrap.cc:654] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:26.363797 15394 tablet_bootstrap.cc:654] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:26.364784 15395 tablet_bootstrap.cc:492] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e: No bootstrap required, opened a new log
I20240501 08:00:26.364866 15395 ts_tablet_manager.cc:1367] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:26.364904 15394 tablet_bootstrap.cc:492] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638: No bootstrap required, opened a new log
I20240501 08:00:26.364962 15394 ts_tablet_manager.cc:1367] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 08:00:26.365288 15394 raft_consensus.cc:348] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:26.365366 15394 raft_consensus.cc:374] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:26.365386 15394 raft_consensus.cc:729] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e019f42a04ca40b491694f86026e3638, State: Initialized, Role: FOLLOWER
I20240501 08:00:26.365334 15395 raft_consensus.cc:348] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:26.365435 15395 raft_consensus.cc:374] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:26.365470 15395 raft_consensus.cc:729] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Initialized, Role: FOLLOWER
I20240501 08:00:26.365494 15394 consensus_queue.cc:260] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:26.365586 15395 consensus_queue.cc:260] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:26.365710 15394 ts_tablet_manager.cc:1398] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:00:26.365873 15395 ts_tablet_manager.cc:1398] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:26.375171 15333 raft_consensus.cc:482] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:26.375306 15333 raft_consensus.cc:504] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.375676 15333 leader_election.cc:288] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573)
I20240501 08:00:26.376210 15266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaebe02e13964ab88e1f6c24a9ce8131" candidate_uuid: "6016871632a84df2abdcc5690f07cb9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e019f42a04ca40b491694f86026e3638" is_pre_election: true
I20240501 08:00:26.376348 15266 raft_consensus.cc:2454] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6016871632a84df2abdcc5690f07cb9e in term 0.
I20240501 08:00:26.376633 14806 leader_election.cc:302] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:26.376834 15333 raft_consensus.cc:2781] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:26.376886 15333 raft_consensus.cc:482] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:26.376912 15333 raft_consensus.cc:3037] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:26.377727 15333 raft_consensus.cc:504] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.378095 15333 leader_election.cc:288] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 election: Requested vote from peers e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573)
I20240501 08:00:26.378572 15266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaebe02e13964ab88e1f6c24a9ce8131" candidate_uuid: "6016871632a84df2abdcc5690f07cb9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e019f42a04ca40b491694f86026e3638"
I20240501 08:00:26.378703 15266 raft_consensus.cc:3037] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:26.379554 15266 raft_consensus.cc:2454] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6016871632a84df2abdcc5690f07cb9e in term 1.
I20240501 08:00:26.379848 14806 leader_election.cc:302] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:26.380059 15333 raft_consensus.cc:2781] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:26.380137 15333 raft_consensus.cc:686] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Becoming Leader. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Running, Role: LEADER
I20240501 08:00:26.380323 15333 consensus_queue.cc:237] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.381886 14723 catalog_manager.cc:5484] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6016871632a84df2abdcc5690f07cb9e (127.0.99.129). New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:26.422289 15391 raft_consensus.cc:482] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:26.422461 15391 raft_consensus.cc:504] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:26.422855 15391 leader_election.cc:288] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:26.423460 14870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f809a3f2420f4e8398cd05529886691b" candidate_uuid: "e019f42a04ca40b491694f86026e3638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6016871632a84df2abdcc5690f07cb9e" is_pre_election: true
I20240501 08:00:26.423614 14870 raft_consensus.cc:2454] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e019f42a04ca40b491694f86026e3638 in term 0.
I20240501 08:00:26.423883 15200 leader_election.cc:302] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:26.424176 15391 raft_consensus.cc:2781] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:26.424239 15391 raft_consensus.cc:482] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:26.424264 15391 raft_consensus.cc:3037] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:26.425081 15391 raft_consensus.cc:504] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:26.425366 15391 leader_election.cc:288] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 election: Requested vote from peers 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:26.425853 14870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f809a3f2420f4e8398cd05529886691b" candidate_uuid: "e019f42a04ca40b491694f86026e3638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6016871632a84df2abdcc5690f07cb9e"
I20240501 08:00:26.425972 14870 raft_consensus.cc:3037] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:26.426755 14870 raft_consensus.cc:2454] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e019f42a04ca40b491694f86026e3638 in term 1.
I20240501 08:00:26.427093 15200 leader_election.cc:302] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:26.427261 15391 raft_consensus.cc:2781] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:26.427369 15391 raft_consensus.cc:686] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Becoming Leader. State: Replica: e019f42a04ca40b491694f86026e3638, State: Running, Role: LEADER
I20240501 08:00:26.427521 15391 consensus_queue.cc:237] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:26.429315 14723 catalog_manager.cc:5484] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 reported cstate change: term changed from 0 to 1, leader changed from <none> to e019f42a04ca40b491694f86026e3638 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:26.600019 15391 raft_consensus.cc:482] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:26.600175 15391 raft_consensus.cc:504] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.600546 15391 leader_election.cc:288] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:26.601097 14870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ce0e31c02ff4701954a6c052fdae018" candidate_uuid: "e019f42a04ca40b491694f86026e3638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6016871632a84df2abdcc5690f07cb9e" is_pre_election: true
I20240501 08:00:26.601228 14870 raft_consensus.cc:2454] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e019f42a04ca40b491694f86026e3638 in term 0.
I20240501 08:00:26.601567 15200 leader_election.cc:302] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:26.601747 15391 raft_consensus.cc:2781] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:26.601802 15391 raft_consensus.cc:482] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:26.601850 15391 raft_consensus.cc:3037] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:26.602669 15391 raft_consensus.cc:504] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.602936 15391 leader_election.cc:288] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 election: Requested vote from peers 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:26.603410 14870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ce0e31c02ff4701954a6c052fdae018" candidate_uuid: "e019f42a04ca40b491694f86026e3638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6016871632a84df2abdcc5690f07cb9e"
I20240501 08:00:26.603518 14870 raft_consensus.cc:3037] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:26.604280 14870 raft_consensus.cc:2454] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e019f42a04ca40b491694f86026e3638 in term 1.
I20240501 08:00:26.604589 15200 leader_election.cc:302] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:26.604749 15391 raft_consensus.cc:2781] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:26.604840 15391 raft_consensus.cc:686] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Becoming Leader. State: Replica: e019f42a04ca40b491694f86026e3638, State: Running, Role: LEADER
I20240501 08:00:26.604981 15391 consensus_queue.cc:237] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.606637 14723 catalog_manager.cc:5484] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 reported cstate change: term changed from 0 to 1, leader changed from <none> to e019f42a04ca40b491694f86026e3638 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: HEALTHY } } }
W20240501 08:00:26.760254 15405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:26.760530 15405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:26.763700 15405 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240501 08:00:26.839854 15391 consensus_queue.cc:1035] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:26.848762 14870 raft_consensus.cc:1261] T 3ce0e31c02ff4701954a6c052fdae018 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:26.849313 15391 consensus_queue.cc:1035] T 3ce0e31c02ff4701954a6c052fdae018 P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:26.851433 15266 raft_consensus.cc:1261] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:26.851747 15333 consensus_queue.cc:1035] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 82.0868 ms
time per row: 0.0410434 ms
I20240501 08:00:26.893095 14728 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34210:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 08:00:26.893668 14728 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_2 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240501 08:00:26.898993 15245 tablet_service.cc:1467] Processing CreateTablet for tablet 599cc1bd079449209998e39e5cfee2a5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=58dc1c9d84704008a42cea20932bda35]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:26.898993 14848 tablet_service.cc:1467] Processing CreateTablet for tablet 599cc1bd079449209998e39e5cfee2a5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=58dc1c9d84704008a42cea20932bda35]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:26.899327 15245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 599cc1bd079449209998e39e5cfee2a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:26.899320 14848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 599cc1bd079449209998e39e5cfee2a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:26.899713 14849 tablet_service.cc:1467] Processing CreateTablet for tablet 8b4a4367803e4f53b42a42344d7db654 (DEFAULT_TABLE table=rebalance_test_table_2 [id=58dc1c9d84704008a42cea20932bda35]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:26.900025 14849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b4a4367803e4f53b42a42344d7db654. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:26.899713 14850 tablet_service.cc:1467] Processing CreateTablet for tablet 1d19a2efa6194a1b9321b7b15b84afb1 (DEFAULT_TABLE table=rebalance_test_table_2 [id=58dc1c9d84704008a42cea20932bda35]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:26.900589 14850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d19a2efa6194a1b9321b7b15b84afb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:26.901592 15453 tablet_bootstrap.cc:492] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638: Bootstrap starting.
I20240501 08:00:26.902290 15244 tablet_service.cc:1467] Processing CreateTablet for tablet 8b4a4367803e4f53b42a42344d7db654 (DEFAULT_TABLE table=rebalance_test_table_2 [id=58dc1c9d84704008a42cea20932bda35]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:26.902529 15453 tablet_bootstrap.cc:654] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:26.902532 15244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b4a4367803e4f53b42a42344d7db654. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:26.902779 15454 tablet_bootstrap.cc:492] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e: Bootstrap starting.
I20240501 08:00:26.902626 15245 tablet_service.cc:1467] Processing CreateTablet for tablet 1d19a2efa6194a1b9321b7b15b84afb1 (DEFAULT_TABLE table=rebalance_test_table_2 [id=58dc1c9d84704008a42cea20932bda35]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:26.902976 15245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d19a2efa6194a1b9321b7b15b84afb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:26.903592 15454 tablet_bootstrap.cc:654] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:26.905023 15454 tablet_bootstrap.cc:492] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e: No bootstrap required, opened a new log
I20240501 08:00:26.905104 15454 ts_tablet_manager.cc:1367] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:26.905164 15453 tablet_bootstrap.cc:492] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638: No bootstrap required, opened a new log
I20240501 08:00:26.905252 15453 ts_tablet_manager.cc:1367] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 08:00:26.905529 15454 raft_consensus.cc:348] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.905625 15454 raft_consensus.cc:374] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:26.905647 15454 raft_consensus.cc:729] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Initialized, Role: FOLLOWER
I20240501 08:00:26.905768 15454 consensus_queue.cc:260] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.905741 15453 raft_consensus.cc:348] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.905845 15453 raft_consensus.cc:374] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:26.905885 15453 raft_consensus.cc:729] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e019f42a04ca40b491694f86026e3638, State: Initialized, Role: FOLLOWER
I20240501 08:00:26.906026 15454 ts_tablet_manager.cc:1398] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:26.906019 15453 consensus_queue.cc:260] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.906270 15453 ts_tablet_manager.cc:1398] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:26.906364 15454 tablet_bootstrap.cc:492] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e: Bootstrap starting.
I20240501 08:00:26.906474 15453 tablet_bootstrap.cc:492] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638: Bootstrap starting.
I20240501 08:00:26.907243 15454 tablet_bootstrap.cc:654] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:26.907325 15453 tablet_bootstrap.cc:654] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:26.908475 15453 tablet_bootstrap.cc:492] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638: No bootstrap required, opened a new log
I20240501 08:00:26.908557 15453 ts_tablet_manager.cc:1367] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:26.908787 15454 tablet_bootstrap.cc:492] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e: No bootstrap required, opened a new log
I20240501 08:00:26.908857 15454 ts_tablet_manager.cc:1367] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 08:00:26.908882 15453 raft_consensus.cc:348] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:26.908962 15453 raft_consensus.cc:374] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:26.908980 15453 raft_consensus.cc:729] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e019f42a04ca40b491694f86026e3638, State: Initialized, Role: FOLLOWER
I20240501 08:00:26.909073 15453 consensus_queue.cc:260] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:26.909320 15453 ts_tablet_manager.cc:1398] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:26.909292 15454 raft_consensus.cc:348] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:26.909374 15454 raft_consensus.cc:374] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:26.909394 15454 raft_consensus.cc:729] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Initialized, Role: FOLLOWER
I20240501 08:00:26.909466 15453 tablet_bootstrap.cc:492] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638: Bootstrap starting.
I20240501 08:00:26.909487 15454 consensus_queue.cc:260] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:26.909762 15454 ts_tablet_manager.cc:1398] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:26.909888 15454 tablet_bootstrap.cc:492] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e: Bootstrap starting.
I20240501 08:00:26.910269 15453 tablet_bootstrap.cc:654] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:26.910684 15454 tablet_bootstrap.cc:654] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:26.911176 15453 tablet_bootstrap.cc:492] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638: No bootstrap required, opened a new log
I20240501 08:00:26.911492 15453 ts_tablet_manager.cc:1367] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:26.912007 15453 raft_consensus.cc:348] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.912170 15453 raft_consensus.cc:374] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:26.912246 15453 raft_consensus.cc:729] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e019f42a04ca40b491694f86026e3638, State: Initialized, Role: FOLLOWER
I20240501 08:00:26.912369 15453 consensus_queue.cc:260] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.912640 15453 ts_tablet_manager.cc:1398] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:26.912902 15454 tablet_bootstrap.cc:492] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e: No bootstrap required, opened a new log
I20240501 08:00:26.912990 15454 ts_tablet_manager.cc:1367] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 08:00:26.913379 15454 raft_consensus.cc:348] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.913483 15454 raft_consensus.cc:374] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:26.913514 15454 raft_consensus.cc:729] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Initialized, Role: FOLLOWER
I20240501 08:00:26.913703 15454 consensus_queue.cc:260] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.913949 15454 ts_tablet_manager.cc:1398] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:26.949755 15333 raft_consensus.cc:482] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:26.949896 15333 raft_consensus.cc:504] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.950264 15333 leader_election.cc:288] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573)
I20240501 08:00:26.950817 15266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d19a2efa6194a1b9321b7b15b84afb1" candidate_uuid: "6016871632a84df2abdcc5690f07cb9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e019f42a04ca40b491694f86026e3638" is_pre_election: true
I20240501 08:00:26.950958 15266 raft_consensus.cc:2454] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6016871632a84df2abdcc5690f07cb9e in term 0.
I20240501 08:00:26.951325 14806 leader_election.cc:302] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:26.951532 15333 raft_consensus.cc:2781] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:26.951622 15333 raft_consensus.cc:482] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:26.951651 15333 raft_consensus.cc:3037] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:26.952594 15333 raft_consensus.cc:504] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.952919 15333 leader_election.cc:288] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 election: Requested vote from peers e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573)
I20240501 08:00:26.953354 15266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d19a2efa6194a1b9321b7b15b84afb1" candidate_uuid: "6016871632a84df2abdcc5690f07cb9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e019f42a04ca40b491694f86026e3638"
I20240501 08:00:26.953469 15266 raft_consensus.cc:3037] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:26.954345 15266 raft_consensus.cc:2454] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6016871632a84df2abdcc5690f07cb9e in term 1.
I20240501 08:00:26.954640 14806 leader_election.cc:302] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:26.954833 15333 raft_consensus.cc:2781] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:26.954941 15333 raft_consensus.cc:686] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Becoming Leader. State: Replica: 6016871632a84df2abdcc5690f07cb9e, State: Running, Role: LEADER
I20240501 08:00:26.955086 15333 consensus_queue.cc:237] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:26.956725 14726 catalog_manager.cc:5484] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6016871632a84df2abdcc5690f07cb9e (127.0.99.129). New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:27.179337 15391 raft_consensus.cc:482] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:27.179479 15391 raft_consensus.cc:504] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:27.179849 15391 leader_election.cc:288] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:27.180444 14869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b4a4367803e4f53b42a42344d7db654" candidate_uuid: "e019f42a04ca40b491694f86026e3638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6016871632a84df2abdcc5690f07cb9e" is_pre_election: true
I20240501 08:00:27.180579 14869 raft_consensus.cc:2454] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e019f42a04ca40b491694f86026e3638 in term 0.
I20240501 08:00:27.181057 15200 leader_election.cc:302] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:27.181314 15391 raft_consensus.cc:2781] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:27.181386 15391 raft_consensus.cc:482] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:27.181418 15391 raft_consensus.cc:3037] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:27.182214 15391 raft_consensus.cc:504] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:27.182525 15391 leader_election.cc:288] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 election: Requested vote from peers 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:27.183044 14869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b4a4367803e4f53b42a42344d7db654" candidate_uuid: "e019f42a04ca40b491694f86026e3638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6016871632a84df2abdcc5690f07cb9e"
I20240501 08:00:27.183174 14869 raft_consensus.cc:3037] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:27.184067 14869 raft_consensus.cc:2454] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e019f42a04ca40b491694f86026e3638 in term 1.
I20240501 08:00:27.184423 15200 leader_election.cc:302] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:27.184635 15391 raft_consensus.cc:2781] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:27.184736 15391 raft_consensus.cc:686] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Becoming Leader. State: Replica: e019f42a04ca40b491694f86026e3638, State: Running, Role: LEADER
I20240501 08:00:27.184875 15391 consensus_queue.cc:237] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } }
I20240501 08:00:27.186715 14726 catalog_manager.cc:5484] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 reported cstate change: term changed from 0 to 1, leader changed from <none> to e019f42a04ca40b491694f86026e3638 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:27.273114 15391 raft_consensus.cc:482] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:27.273253 15391 raft_consensus.cc:504] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:27.273607 15391 leader_election.cc:288] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:27.274173 14869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "599cc1bd079449209998e39e5cfee2a5" candidate_uuid: "e019f42a04ca40b491694f86026e3638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6016871632a84df2abdcc5690f07cb9e" is_pre_election: true
I20240501 08:00:27.274323 14869 raft_consensus.cc:2454] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e019f42a04ca40b491694f86026e3638 in term 0.
I20240501 08:00:27.274652 15200 leader_election.cc:302] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:27.274817 15391 raft_consensus.cc:2781] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:27.274873 15391 raft_consensus.cc:482] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:27.274909 15391 raft_consensus.cc:3037] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:27.275738 15391 raft_consensus.cc:504] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:27.276049 15391 leader_election.cc:288] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 election: Requested vote from peers 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:27.276393 14869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "599cc1bd079449209998e39e5cfee2a5" candidate_uuid: "e019f42a04ca40b491694f86026e3638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6016871632a84df2abdcc5690f07cb9e"
I20240501 08:00:27.276494 14869 raft_consensus.cc:3037] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:27.277215 14869 raft_consensus.cc:2454] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e019f42a04ca40b491694f86026e3638 in term 1.
I20240501 08:00:27.277490 15200 leader_election.cc:302] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 6016871632a84df2abdcc5690f07cb9e, e019f42a04ca40b491694f86026e3638; no voters:
I20240501 08:00:27.277639 15391 raft_consensus.cc:2781] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:27.277719 15391 raft_consensus.cc:686] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Becoming Leader. State: Replica: e019f42a04ca40b491694f86026e3638, State: Running, Role: LEADER
I20240501 08:00:27.277853 15391 consensus_queue.cc:237] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } }
I20240501 08:00:27.279532 14726 catalog_manager.cc:5484] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 reported cstate change: term changed from 0 to 1, leader changed from <none> to e019f42a04ca40b491694f86026e3638 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: HEALTHY } } }
W20240501 08:00:27.462363 15464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:27.462623 15464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:27.465940 15464 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 08:00:27.468724 15333 consensus_queue.cc:1035] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_2'
I20240501 08:00:27.554390 14869 raft_consensus.cc:1261] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:27.554862 15391 consensus_queue.cc:1035] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:27.562140 14869 raft_consensus.cc:1261] T 8b4a4367803e4f53b42a42344d7db654 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:27.562773 15391 consensus_queue.cc:1035] T 8b4a4367803e4f53b42a42344d7db654 P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 90.9709 ms
time per row: 0.0454855 ms
I20240501 08:00:27.601145 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:41275
--local_ip_for_outbound_sockets=127.0.99.130
--tserver_master_addrs=127.0.99.190:40571
--webserver_port=40725
--webserver_interface=127.0.99.130
--builtin_ntp_servers=127.0.99.148:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 08:00:27.711190 15510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:27.711437 15510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:27.711485 15510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:27.714697 15510 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:27.714780 15510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:27.714864 15510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240501 08:00:27.718714 15510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:41275
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=40725
--tserver_master_addrs=127.0.99.190:40571
--never_fsync=true
--heap_profile_path=/tmp/kudu.15510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:27.719672 15510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:27.721021 15510 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:27.730679 15518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:27.731606 15515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:27.732204 15516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:27.732947 15510 server_base.cc:1034] running on GCE node
I20240501 08:00:27.733417 15510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:27.734130 15510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:27.735324 15510 hybrid_clock.cc:648] HybridClock initialized: now 1714550427735288 us; error 53 us; skew 500 ppm
I20240501 08:00:27.737447 15510 webserver.cc:460] Webserver started at http://127.0.99.130:40725/ using document root <none> and password file <none>
I20240501 08:00:27.738011 15510 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:27.738086 15510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:27.741706 15510 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 08:00:27.743918 15524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:27.745263 15510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:27.745407 15510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "58593562dc504df98fbba7f0b734958f"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:27.745995 15510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:27.774478 15510 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:27.775132 15510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:27.775298 15510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:27.775844 15510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:27.776773 15510 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:27.776822 15510 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:27.776875 15510 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:27.776902 15510 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:27.787036 15510 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:41275
I20240501 08:00:27.787061 15637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:41275 every 8 connection(s)
I20240501 08:00:27.788297 15510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 08:00:27.797271 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 15510
I20240501 08:00:27.797762 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:34069
--local_ip_for_outbound_sockets=127.0.99.131
--tserver_master_addrs=127.0.99.190:40571
--webserver_port=45361
--webserver_interface=127.0.99.131
--builtin_ntp_servers=127.0.99.148:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:27.797835 15638 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40571
I20240501 08:00:27.798071 15638 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:27.798676 15638 heartbeater.cc:511] Master 127.0.99.190:40571 requested a full tablet report, sending...
I20240501 08:00:27.799875 14724 ts_manager.cc:194] Re-registered known tserver with Master: 58593562dc504df98fbba7f0b734958f (127.0.99.130:41275)
I20240501 08:00:27.800496 14724 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:51713
W20240501 08:00:27.909034 15641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:27.909273 15641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:27.909317 15641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:27.912449 15641 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:27.912520 15641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:27.912601 15641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240501 08:00:27.916415 15641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:34069
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=45361
--tserver_master_addrs=127.0.99.190:40571
--never_fsync=true
--heap_profile_path=/tmp/kudu.15641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:27.917374 15641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:27.918499 15641 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:27.927420 15646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:27.928349 15649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:27.929095 15647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:27.929744 15641 server_base.cc:1034] running on GCE node
I20240501 08:00:27.930218 15641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:27.931072 15641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:27.932242 15641 hybrid_clock.cc:648] HybridClock initialized: now 1714550427932228 us; error 36 us; skew 500 ppm
I20240501 08:00:27.934361 15641 webserver.cc:460] Webserver started at http://127.0.99.131:45361/ using document root <none> and password file <none>
I20240501 08:00:27.935015 15641 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:27.935102 15641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:27.939366 15641 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:27.941766 15655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:27.943002 15641 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:27.943128 15641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff"
format_stamp: "Formatted at 2024-05-01 08:00:21 on dist-test-slave-f81v"
I20240501 08:00:27.943691 15641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:27.958165 15641 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:27.958804 15641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:27.958979 15641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:27.959627 15641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:27.960597 15641 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:27.960641 15641 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:27.960680 15641 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:27.960693 15641 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:27.969601 15641 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:34069
I20240501 08:00:27.969627 15768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:34069 every 8 connection(s)
I20240501 08:00:27.970629 15641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 08:00:27.974449 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 15641
I20240501 08:00:27.981554 15769 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40571
I20240501 08:00:27.981796 15769 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:27.982489 15769 heartbeater.cc:511] Master 127.0.99.190:40571 requested a full tablet report, sending...
I20240501 08:00:27.983664 14724 ts_manager.cc:194] Re-registered known tserver with Master: 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131:34069)
I20240501 08:00:27.984266 14724 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:45017
I20240501 08:00:28.171622 14850 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:28.173218 15572 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:28.173476 15703 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:28.173744 15245 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:28.269495 14850 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:28.269665 15572 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:28.272367 15703 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:28.281508 15245 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:28.382086 15572 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:28.382717 14850 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:28.384407 15245 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:28.391844 15703 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:28.475425 15703 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:28.475564 15572 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:28.476753 14850 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:28.480967 15245 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:28.512557 15266 consensus_queue.cc:237] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } }
I20240501 08:00:28.514228 14869 raft_consensus.cc:1261] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:28.514699 15911 consensus_queue.cc:1035] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:28.517042 15381 raft_consensus.cc:2932] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 58593562dc504df98fbba7f0b734958f (127.0.99.130) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } } }
I20240501 08:00:28.517565 14869 raft_consensus.cc:2932] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 58593562dc504df98fbba7f0b734958f (127.0.99.130) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } } }
W20240501 08:00:28.520079 15201 consensus_peers.cc:485] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 -> Peer 58593562dc504df98fbba7f0b734958f (127.0.99.130:41275): Couldn't send request to peer 58593562dc504df98fbba7f0b734958f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eb631e0b15744ca4b5f767f06711a3a7. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:28.520263 14724 catalog_manager.cc:5484] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: config changed from index -1 to 4, NON_VOTER 58593562dc504df98fbba7f0b734958f (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } } }
I20240501 08:00:28.529713 14869 consensus_queue.cc:237] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } }
I20240501 08:00:28.531461 15266 raft_consensus.cc:1261] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:28.531920 15469 consensus_queue.cc:1035] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:28.533802 15469 raft_consensus.cc:2932] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } } }
I20240501 08:00:28.534150 15266 raft_consensus.cc:2932] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } } }
W20240501 08:00:28.535288 14806 consensus_peers.cc:485] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e -> Peer 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131:34069): Couldn't send request to peer 7a8a3e5fb2c14eb9986d9da6cb1229ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d19a2efa6194a1b9321b7b15b84afb1. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:28.537021 14726 catalog_manager.cc:5484] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 reported cstate change: config changed from index -1 to 4, NON_VOTER 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } } }
I20240501 08:00:28.543992 15266 consensus_queue.cc:237] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } }
I20240501 08:00:28.545289 14870 raft_consensus.cc:1261] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 08:00:28.545375 15201 consensus_peers.cc:485] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 -> Peer 58593562dc504df98fbba7f0b734958f (127.0.99.130:41275): Couldn't send request to peer 58593562dc504df98fbba7f0b734958f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 599cc1bd079449209998e39e5cfee2a5. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:28.545878 15911 consensus_queue.cc:1035] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:28.547448 15381 raft_consensus.cc:2932] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 58593562dc504df98fbba7f0b734958f (127.0.99.130) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } } }
I20240501 08:00:28.547843 14869 raft_consensus.cc:2932] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 58593562dc504df98fbba7f0b734958f (127.0.99.130) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } } }
I20240501 08:00:28.550149 14724 catalog_manager.cc:5484] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 reported cstate change: config changed from index -1 to 4, NON_VOTER 58593562dc504df98fbba7f0b734958f (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:28.558266 14869 consensus_queue.cc:237] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } }
I20240501 08:00:28.559621 15266 raft_consensus.cc:1261] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 08:00:28.559875 14806 consensus_peers.cc:485] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e -> Peer 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131:34069): Couldn't send request to peer 7a8a3e5fb2c14eb9986d9da6cb1229ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ed99c9a3b7542eca1199ff105198318. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:28.560098 15926 consensus_queue.cc:1035] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:28.562078 15924 raft_consensus.cc:2932] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } } }
I20240501 08:00:28.562582 15266 raft_consensus.cc:2932] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } } }
I20240501 08:00:28.564941 14726 catalog_manager.cc:5484] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: config changed from index -1 to 4, NON_VOTER 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:28.573262 15266 consensus_queue.cc:237] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } }
I20240501 08:00:28.574738 14869 raft_consensus.cc:1261] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:28.575139 15911 consensus_queue.cc:1035] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:28.577286 15911 raft_consensus.cc:2932] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } } }
I20240501 08:00:28.577913 14869 raft_consensus.cc:2932] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } } }
W20240501 08:00:28.578164 15202 consensus_peers.cc:485] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 -> Peer 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131:34069): Couldn't send request to peer 7a8a3e5fb2c14eb9986d9da6cb1229ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f809a3f2420f4e8398cd05529886691b. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:28.580493 14724 catalog_manager.cc:5484] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e reported cstate change: config changed from index -1 to 4, NON_VOTER 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } } }
I20240501 08:00:28.588827 14869 consensus_queue.cc:237] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } }
I20240501 08:00:28.590824 15266 raft_consensus.cc:1261] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:28.591289 15926 consensus_queue.cc:1035] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:28.593302 15924 raft_consensus.cc:2932] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 58593562dc504df98fbba7f0b734958f (127.0.99.130) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } } }
W20240501 08:00:28.593901 14805 consensus_peers.cc:485] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e -> Peer 58593562dc504df98fbba7f0b734958f (127.0.99.130:41275): Couldn't send request to peer 58593562dc504df98fbba7f0b734958f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aaebe02e13964ab88e1f6c24a9ce8131. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:28.593783 15266 raft_consensus.cc:2932] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 58593562dc504df98fbba7f0b734958f (127.0.99.130) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } } }
I20240501 08:00:28.596009 14721 catalog_manager.cc:5484] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: config changed from index -1 to 4, NON_VOTER 58593562dc504df98fbba7f0b734958f (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:28.802544 15638 heartbeater.cc:503] Master 127.0.99.190:40571 was elected leader, sending a full tablet report...
I20240501 08:00:28.940171 16013 ts_tablet_manager.cc:902] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Initiating tablet copy from peer 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:28.940896 16013 tablet_copy_client.cc:308] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:39697
I20240501 08:00:28.945356 14890 tablet_copy_service.cc:140] P 6016871632a84df2abdcc5690f07cb9e: Received BeginTabletCopySession request for tablet 3ed99c9a3b7542eca1199ff105198318 from peer 7a8a3e5fb2c14eb9986d9da6cb1229ff ({username='slave'} at 127.0.99.131:41629)
I20240501 08:00:28.945524 14890 tablet_copy_service.cc:161] P 6016871632a84df2abdcc5690f07cb9e: Beginning new tablet copy session on tablet 3ed99c9a3b7542eca1199ff105198318 from peer 7a8a3e5fb2c14eb9986d9da6cb1229ff at {username='slave'} at 127.0.99.131:41629: session id = 7a8a3e5fb2c14eb9986d9da6cb1229ff-3ed99c9a3b7542eca1199ff105198318
I20240501 08:00:28.946718 14890 tablet_copy_source_session.cc:205] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:28.950287 16013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ed99c9a3b7542eca1199ff105198318. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:28.955920 16013 tablet_copy_client.cc:783] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:28.956140 16013 tablet_copy_client.cc:647] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:28.958285 16013 tablet_copy_client.cc:523] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:28.960003 16013 tablet_bootstrap.cc:492] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Bootstrap starting.
I20240501 08:00:28.965396 16013 log.cc:824] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:28.984827 16013 tablet_bootstrap.cc:492] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:28.985211 16013 tablet_bootstrap.cc:492] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Bootstrap complete.
I20240501 08:00:28.985522 16013 ts_tablet_manager.cc:1367] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Time spent bootstrapping tablet: real 0.026s user 0.025s sys 0.000s
I20240501 08:00:28.986217 15769 heartbeater.cc:503] Master 127.0.99.190:40571 was elected leader, sending a full tablet report...
I20240501 08:00:28.987649 16013 raft_consensus.cc:348] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } }
I20240501 08:00:28.987890 16013 raft_consensus.cc:729] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a8a3e5fb2c14eb9986d9da6cb1229ff, State: Initialized, Role: LEARNER
I20240501 08:00:28.988236 16013 consensus_queue.cc:260] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } }
I20240501 08:00:28.989084 16013 ts_tablet_manager.cc:1398] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 08:00:28.989773 14890 tablet_copy_service.cc:342] P 6016871632a84df2abdcc5690f07cb9e: Request end of tablet copy session 7a8a3e5fb2c14eb9986d9da6cb1229ff-3ed99c9a3b7542eca1199ff105198318 received from {username='slave'} at 127.0.99.131:41629
I20240501 08:00:28.989864 14890 tablet_copy_service.cc:433] P 6016871632a84df2abdcc5690f07cb9e: ending tablet copy session 7a8a3e5fb2c14eb9986d9da6cb1229ff-3ed99c9a3b7542eca1199ff105198318 on tablet 3ed99c9a3b7542eca1199ff105198318 with peer 7a8a3e5fb2c14eb9986d9da6cb1229ff
I20240501 08:00:29.004025 16021 ts_tablet_manager.cc:902] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f: Initiating tablet copy from peer e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573)
I20240501 08:00:29.004758 16021 tablet_copy_client.cc:308] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:40573
I20240501 08:00:29.008899 15286 tablet_copy_service.cc:140] P e019f42a04ca40b491694f86026e3638: Received BeginTabletCopySession request for tablet eb631e0b15744ca4b5f767f06711a3a7 from peer 58593562dc504df98fbba7f0b734958f ({username='slave'} at 127.0.99.130:34207)
I20240501 08:00:29.009073 15286 tablet_copy_service.cc:161] P e019f42a04ca40b491694f86026e3638: Beginning new tablet copy session on tablet eb631e0b15744ca4b5f767f06711a3a7 from peer 58593562dc504df98fbba7f0b734958f at {username='slave'} at 127.0.99.130:34207: session id = 58593562dc504df98fbba7f0b734958f-eb631e0b15744ca4b5f767f06711a3a7
I20240501 08:00:29.010496 15286 tablet_copy_source_session.cc:205] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:29.014003 16021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb631e0b15744ca4b5f767f06711a3a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:29.020287 16021 tablet_copy_client.cc:783] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:29.020514 16021 tablet_copy_client.cc:647] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:29.022825 16021 tablet_copy_client.cc:523] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:29.024739 16021 tablet_bootstrap.cc:492] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f: Bootstrap starting.
I20240501 08:00:29.030546 16021 log.cc:824] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:29.050621 16021 tablet_bootstrap.cc:492] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:29.051009 16021 tablet_bootstrap.cc:492] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f: Bootstrap complete.
I20240501 08:00:29.051312 16021 ts_tablet_manager.cc:1367] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f: Time spent bootstrapping tablet: real 0.027s user 0.024s sys 0.000s
I20240501 08:00:29.053939 16021 raft_consensus.cc:348] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } }
I20240501 08:00:29.054177 16021 raft_consensus.cc:729] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58593562dc504df98fbba7f0b734958f, State: Initialized, Role: LEARNER
I20240501 08:00:29.054502 16021 consensus_queue.cc:260] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } }
I20240501 08:00:29.055315 16021 ts_tablet_manager.cc:1398] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20240501 08:00:29.056142 15286 tablet_copy_service.cc:342] P e019f42a04ca40b491694f86026e3638: Request end of tablet copy session 58593562dc504df98fbba7f0b734958f-eb631e0b15744ca4b5f767f06711a3a7 received from {username='slave'} at 127.0.99.130:34207
I20240501 08:00:29.056246 15286 tablet_copy_service.cc:433] P e019f42a04ca40b491694f86026e3638: ending tablet copy session 58593562dc504df98fbba7f0b734958f-eb631e0b15744ca4b5f767f06711a3a7 on tablet eb631e0b15744ca4b5f767f06711a3a7 with peer 58593562dc504df98fbba7f0b734958f
I20240501 08:00:29.056849 16013 ts_tablet_manager.cc:902] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Initiating tablet copy from peer e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573)
I20240501 08:00:29.057359 16013 tablet_copy_client.cc:308] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:40573
I20240501 08:00:29.060686 15286 tablet_copy_service.cc:140] P e019f42a04ca40b491694f86026e3638: Received BeginTabletCopySession request for tablet f809a3f2420f4e8398cd05529886691b from peer 7a8a3e5fb2c14eb9986d9da6cb1229ff ({username='slave'} at 127.0.99.131:39787)
I20240501 08:00:29.060789 15286 tablet_copy_service.cc:161] P e019f42a04ca40b491694f86026e3638: Beginning new tablet copy session on tablet f809a3f2420f4e8398cd05529886691b from peer 7a8a3e5fb2c14eb9986d9da6cb1229ff at {username='slave'} at 127.0.99.131:39787: session id = 7a8a3e5fb2c14eb9986d9da6cb1229ff-f809a3f2420f4e8398cd05529886691b
I20240501 08:00:29.061514 15286 tablet_copy_source_session.cc:205] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:29.062314 16013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f809a3f2420f4e8398cd05529886691b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:29.064231 16013 tablet_copy_client.cc:783] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:29.064391 16013 tablet_copy_client.cc:647] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:29.065752 16013 tablet_copy_client.cc:523] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:29.066881 16013 tablet_bootstrap.cc:492] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Bootstrap starting.
I20240501 08:00:29.089166 16013 tablet_bootstrap.cc:492] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:29.089473 16013 tablet_bootstrap.cc:492] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Bootstrap complete.
I20240501 08:00:29.089671 16013 ts_tablet_manager.cc:1367] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Time spent bootstrapping tablet: real 0.023s user 0.022s sys 0.000s
I20240501 08:00:29.090306 16013 raft_consensus.cc:348] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } }
I20240501 08:00:29.090427 16013 raft_consensus.cc:729] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a8a3e5fb2c14eb9986d9da6cb1229ff, State: Initialized, Role: LEARNER
I20240501 08:00:29.090538 16013 consensus_queue.cc:260] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } }
I20240501 08:00:29.090842 16013 ts_tablet_manager.cc:1398] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 08:00:29.091399 15286 tablet_copy_service.cc:342] P e019f42a04ca40b491694f86026e3638: Request end of tablet copy session 7a8a3e5fb2c14eb9986d9da6cb1229ff-f809a3f2420f4e8398cd05529886691b received from {username='slave'} at 127.0.99.131:39787
I20240501 08:00:29.091475 15286 tablet_copy_service.cc:433] P e019f42a04ca40b491694f86026e3638: ending tablet copy session 7a8a3e5fb2c14eb9986d9da6cb1229ff-f809a3f2420f4e8398cd05529886691b on tablet f809a3f2420f4e8398cd05529886691b with peer 7a8a3e5fb2c14eb9986d9da6cb1229ff
I20240501 08:00:29.127218 16021 ts_tablet_manager.cc:902] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f: Initiating tablet copy from peer e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573)
I20240501 08:00:29.127743 16021 tablet_copy_client.cc:308] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:40573
I20240501 08:00:29.128497 15286 tablet_copy_service.cc:140] P e019f42a04ca40b491694f86026e3638: Received BeginTabletCopySession request for tablet 599cc1bd079449209998e39e5cfee2a5 from peer 58593562dc504df98fbba7f0b734958f ({username='slave'} at 127.0.99.130:34207)
I20240501 08:00:29.128638 15286 tablet_copy_service.cc:161] P e019f42a04ca40b491694f86026e3638: Beginning new tablet copy session on tablet 599cc1bd079449209998e39e5cfee2a5 from peer 58593562dc504df98fbba7f0b734958f at {username='slave'} at 127.0.99.130:34207: session id = 58593562dc504df98fbba7f0b734958f-599cc1bd079449209998e39e5cfee2a5
I20240501 08:00:29.129384 15286 tablet_copy_source_session.cc:205] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:29.130308 16021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 599cc1bd079449209998e39e5cfee2a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:29.132225 16021 tablet_copy_client.cc:783] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:29.132375 16021 tablet_copy_client.cc:647] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:29.133621 16021 tablet_copy_client.cc:523] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:29.134704 16021 tablet_bootstrap.cc:492] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f: Bootstrap starting.
I20240501 08:00:29.139658 16013 ts_tablet_manager.cc:902] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Initiating tablet copy from peer 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:29.140126 16013 tablet_copy_client.cc:308] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:39697
I20240501 08:00:29.140604 14890 tablet_copy_service.cc:140] P 6016871632a84df2abdcc5690f07cb9e: Received BeginTabletCopySession request for tablet 1d19a2efa6194a1b9321b7b15b84afb1 from peer 7a8a3e5fb2c14eb9986d9da6cb1229ff ({username='slave'} at 127.0.99.131:41629)
I20240501 08:00:29.140866 14890 tablet_copy_service.cc:161] P 6016871632a84df2abdcc5690f07cb9e: Beginning new tablet copy session on tablet 1d19a2efa6194a1b9321b7b15b84afb1 from peer 7a8a3e5fb2c14eb9986d9da6cb1229ff at {username='slave'} at 127.0.99.131:41629: session id = 7a8a3e5fb2c14eb9986d9da6cb1229ff-1d19a2efa6194a1b9321b7b15b84afb1
I20240501 08:00:29.141942 14890 tablet_copy_source_session.cc:205] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:29.143325 16013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d19a2efa6194a1b9321b7b15b84afb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:29.145931 16013 tablet_copy_client.cc:783] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:29.146104 16013 tablet_copy_client.cc:647] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:29.147739 16013 tablet_copy_client.cc:523] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:29.149336 16013 tablet_bootstrap.cc:492] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Bootstrap starting.
I20240501 08:00:29.165580 16021 tablet_bootstrap.cc:492] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:29.165912 16021 tablet_bootstrap.cc:492] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f: Bootstrap complete.
I20240501 08:00:29.166108 16021 ts_tablet_manager.cc:1367] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f: Time spent bootstrapping tablet: real 0.031s user 0.029s sys 0.000s
I20240501 08:00:29.167160 16021 raft_consensus.cc:348] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } }
I20240501 08:00:29.167312 16021 raft_consensus.cc:729] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58593562dc504df98fbba7f0b734958f, State: Initialized, Role: LEARNER
I20240501 08:00:29.167479 16021 consensus_queue.cc:260] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } }
I20240501 08:00:29.168633 16021 ts_tablet_manager.cc:1398] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:29.169235 15286 tablet_copy_service.cc:342] P e019f42a04ca40b491694f86026e3638: Request end of tablet copy session 58593562dc504df98fbba7f0b734958f-599cc1bd079449209998e39e5cfee2a5 received from {username='slave'} at 127.0.99.130:34207
I20240501 08:00:29.169322 15286 tablet_copy_service.cc:433] P e019f42a04ca40b491694f86026e3638: ending tablet copy session 58593562dc504df98fbba7f0b734958f-599cc1bd079449209998e39e5cfee2a5 on tablet 599cc1bd079449209998e39e5cfee2a5 with peer 58593562dc504df98fbba7f0b734958f
I20240501 08:00:29.170665 16021 ts_tablet_manager.cc:902] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f: Initiating tablet copy from peer 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697)
I20240501 08:00:29.171100 16021 tablet_copy_client.cc:308] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:39697
I20240501 08:00:29.175349 14890 tablet_copy_service.cc:140] P 6016871632a84df2abdcc5690f07cb9e: Received BeginTabletCopySession request for tablet aaebe02e13964ab88e1f6c24a9ce8131 from peer 58593562dc504df98fbba7f0b734958f ({username='slave'} at 127.0.99.130:45749)
I20240501 08:00:29.175473 16013 tablet_bootstrap.cc:492] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:29.175501 14890 tablet_copy_service.cc:161] P 6016871632a84df2abdcc5690f07cb9e: Beginning new tablet copy session on tablet aaebe02e13964ab88e1f6c24a9ce8131 from peer 58593562dc504df98fbba7f0b734958f at {username='slave'} at 127.0.99.130:45749: session id = 58593562dc504df98fbba7f0b734958f-aaebe02e13964ab88e1f6c24a9ce8131
I20240501 08:00:29.175733 16013 tablet_bootstrap.cc:492] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Bootstrap complete.
I20240501 08:00:29.175964 16013 ts_tablet_manager.cc:1367] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Time spent bootstrapping tablet: real 0.027s user 0.027s sys 0.000s
I20240501 08:00:29.176493 14890 tablet_copy_source_session.cc:205] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:29.176478 16013 raft_consensus.cc:348] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } }
I20240501 08:00:29.176611 16013 raft_consensus.cc:729] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a8a3e5fb2c14eb9986d9da6cb1229ff, State: Initialized, Role: LEARNER
I20240501 08:00:29.176717 16013 consensus_queue.cc:260] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: true } }
I20240501 08:00:29.177011 16013 ts_tablet_manager.cc:1398] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:29.177563 14890 tablet_copy_service.cc:342] P 6016871632a84df2abdcc5690f07cb9e: Request end of tablet copy session 7a8a3e5fb2c14eb9986d9da6cb1229ff-1d19a2efa6194a1b9321b7b15b84afb1 received from {username='slave'} at 127.0.99.131:41629
I20240501 08:00:29.177639 14890 tablet_copy_service.cc:433] P 6016871632a84df2abdcc5690f07cb9e: ending tablet copy session 7a8a3e5fb2c14eb9986d9da6cb1229ff-1d19a2efa6194a1b9321b7b15b84afb1 on tablet 1d19a2efa6194a1b9321b7b15b84afb1 with peer 7a8a3e5fb2c14eb9986d9da6cb1229ff
I20240501 08:00:29.177883 16021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaebe02e13964ab88e1f6c24a9ce8131. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:29.180058 16021 tablet_copy_client.cc:783] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:29.180222 16021 tablet_copy_client.cc:647] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:29.181843 16021 tablet_copy_client.cc:523] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:29.183280 16021 tablet_bootstrap.cc:492] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f: Bootstrap starting.
I20240501 08:00:29.200424 16021 tablet_bootstrap.cc:492] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:29.200699 16021 tablet_bootstrap.cc:492] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f: Bootstrap complete.
I20240501 08:00:29.200855 16021 ts_tablet_manager.cc:1367] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f: Time spent bootstrapping tablet: real 0.018s user 0.018s sys 0.000s
I20240501 08:00:29.201282 16021 raft_consensus.cc:348] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } }
I20240501 08:00:29.201388 16021 raft_consensus.cc:729] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 58593562dc504df98fbba7f0b734958f, State: Initialized, Role: LEARNER
I20240501 08:00:29.201462 16021 consensus_queue.cc:260] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: NON_VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: true } }
I20240501 08:00:29.201712 16021 ts_tablet_manager.cc:1398] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:29.202432 14890 tablet_copy_service.cc:342] P 6016871632a84df2abdcc5690f07cb9e: Request end of tablet copy session 58593562dc504df98fbba7f0b734958f-aaebe02e13964ab88e1f6c24a9ce8131 received from {username='slave'} at 127.0.99.130:45749
I20240501 08:00:29.202587 14890 tablet_copy_service.cc:433] P 6016871632a84df2abdcc5690f07cb9e: ending tablet copy session 58593562dc504df98fbba7f0b734958f-aaebe02e13964ab88e1f6c24a9ce8131 on tablet aaebe02e13964ab88e1f6c24a9ce8131 with peer 58593562dc504df98fbba7f0b734958f
I20240501 08:00:29.433326 15723 raft_consensus.cc:1203] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:29.503861 15592 raft_consensus.cc:1203] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:29.546535 15723 raft_consensus.cc:1203] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:29.560217 15592 raft_consensus.cc:1203] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:29.715950 15592 raft_consensus.cc:1203] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:29.753008 15723 raft_consensus.cc:1203] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:29.939172 16111 raft_consensus.cc:1050] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e: attempting to promote NON_VOTER 7a8a3e5fb2c14eb9986d9da6cb1229ff to VOTER
I20240501 08:00:29.939606 16111 consensus_queue.cc:237] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } }
I20240501 08:00:29.941025 15266 raft_consensus.cc:1261] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:29.941045 15723 raft_consensus.cc:1261] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 LEARNER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:29.941421 16111 consensus_queue.cc:1035] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:29.941651 16069 consensus_queue.cc:1035] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:29.943814 16111 raft_consensus.cc:2932] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:29.945576 15266 raft_consensus.cc:2932] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:29.945844 15723 raft_consensus.cc:2932] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:29.947867 14721 catalog_manager.cc:5484] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 reported cstate change: config changed from index 4 to 5, 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:29.952651 14870 consensus_queue.cc:237] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } }
I20240501 08:00:29.953851 15723 raft_consensus.cc:1261] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 FOLLOWER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:29.954277 16069 consensus_queue.cc:1035] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:29.956336 15924 raft_consensus.cc:2932] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e019f42a04ca40b491694f86026e3638 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:29.956908 15723 raft_consensus.cc:2932] T 3ed99c9a3b7542eca1199ff105198318 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e019f42a04ca40b491694f86026e3638 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:29.957842 16113 raft_consensus.cc:1050] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638: attempting to promote NON_VOTER 58593562dc504df98fbba7f0b734958f to VOTER
I20240501 08:00:29.958361 14713 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3ed99c9a3b7542eca1199ff105198318 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:29.958356 16113 consensus_queue.cc:237] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } }
I20240501 08:00:29.959354 14726 catalog_manager.cc:5484] T 3ed99c9a3b7542eca1199ff105198318 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: config changed from index 5 to 6, VOTER e019f42a04ca40b491694f86026e3638 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:29.959722 14870 raft_consensus.cc:1261] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:29.960036 15592 raft_consensus.cc:1261] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f [term 1 LEARNER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:29.960536 16112 consensus_queue.cc:1035] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:29.960821 16113 consensus_queue.cc:1035] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:29.962885 15245 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ed99c9a3b7542eca1199ff105198318 with delete_type TABLET_DATA_TOMBSTONED (TS e019f42a04ca40b491694f86026e3638 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:51886
I20240501 08:00:29.963397 16184 tablet_replica.cc:330] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638: stopping tablet replica
I20240501 08:00:29.963745 16184 raft_consensus.cc:2229] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:29.963681 16113 raft_consensus.cc:2932] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 58593562dc504df98fbba7f0b734958f (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:29.964001 16184 raft_consensus.cc:2258] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:29.964428 14870 raft_consensus.cc:2932] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 58593562dc504df98fbba7f0b734958f (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:29.966647 16184 ts_tablet_manager.cc:1875] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:29.967711 14726 catalog_manager.cc:5484] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: config changed from index 4 to 5, 58593562dc504df98fbba7f0b734958f (127.0.99.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:29.967399 15592 raft_consensus.cc:2932] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 58593562dc504df98fbba7f0b734958f (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:29.972306 16184 ts_tablet_manager.cc:1888] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:29.972436 16184 log.cc:1196] T 3ed99c9a3b7542eca1199ff105198318 P e019f42a04ca40b491694f86026e3638: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/3ed99c9a3b7542eca1199ff105198318
I20240501 08:00:29.973037 14715 catalog_manager.cc:4830] TS e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573): tablet 3ed99c9a3b7542eca1199ff105198318 (table rebalance_test_table_0 [id=6f7b30c496af4f26b0be4a9f8e72ada4]) successfully deleted
I20240501 08:00:29.980293 15266 consensus_queue.cc:237] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } }
I20240501 08:00:29.981588 15592 raft_consensus.cc:1261] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:29.982115 16113 consensus_queue.cc:1035] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:29.984503 16112 raft_consensus.cc:2932] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6016871632a84df2abdcc5690f07cb9e (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:29.985085 15592 raft_consensus.cc:2932] T eb631e0b15744ca4b5f767f06711a3a7 P 58593562dc504df98fbba7f0b734958f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6016871632a84df2abdcc5690f07cb9e (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:29.987169 14715 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet eb631e0b15744ca4b5f767f06711a3a7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:29.987699 14721 catalog_manager.cc:5484] T eb631e0b15744ca4b5f767f06711a3a7 P e019f42a04ca40b491694f86026e3638 reported cstate change: config changed from index 5 to 6, VOTER 6016871632a84df2abdcc5690f07cb9e (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:29.990897 14850 tablet_service.cc:1514] Processing DeleteTablet for tablet eb631e0b15744ca4b5f767f06711a3a7 with delete_type TABLET_DATA_TOMBSTONED (TS 6016871632a84df2abdcc5690f07cb9e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38758
I20240501 08:00:29.991406 16185 tablet_replica.cc:330] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e: stopping tablet replica
I20240501 08:00:29.991750 16185 raft_consensus.cc:2229] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:29.992058 16185 raft_consensus.cc:2258] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:29.993266 16185 ts_tablet_manager.cc:1875] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:29.995251 16185 ts_tablet_manager.cc:1888] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:29.995337 16185 log.cc:1196] T eb631e0b15744ca4b5f767f06711a3a7 P 6016871632a84df2abdcc5690f07cb9e: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/eb631e0b15744ca4b5f767f06711a3a7
I20240501 08:00:29.995976 14713 catalog_manager.cc:4830] TS 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697): tablet eb631e0b15744ca4b5f767f06711a3a7 (table rebalance_test_table_0 [id=6f7b30c496af4f26b0be4a9f8e72ada4]) successfully deleted
I20240501 08:00:30.024662 15381 raft_consensus.cc:1050] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638: attempting to promote NON_VOTER 58593562dc504df98fbba7f0b734958f to VOTER
I20240501 08:00:30.024804 16186 raft_consensus.cc:1050] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638: attempting to promote NON_VOTER 7a8a3e5fb2c14eb9986d9da6cb1229ff to VOTER
I20240501 08:00:30.025079 15381 consensus_queue.cc:237] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } }
I20240501 08:00:30.025122 16186 consensus_queue.cc:237] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } }
I20240501 08:00:30.026479 15592 raft_consensus.cc:1261] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f [term 1 LEARNER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:30.026885 15723 raft_consensus.cc:1261] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 LEARNER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:30.026975 14870 raft_consensus.cc:1261] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:30.027058 14869 raft_consensus.cc:1261] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:30.027324 16113 consensus_queue.cc:1035] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:30.027508 16186 consensus_queue.cc:1035] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:30.027412 16189 consensus_queue.cc:1035] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:30.027982 16190 consensus_queue.cc:1035] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:30.029633 16190 raft_consensus.cc:2932] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 58593562dc504df98fbba7f0b734958f (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:30.031075 16186 raft_consensus.cc:2932] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:30.031530 15592 raft_consensus.cc:2932] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 58593562dc504df98fbba7f0b734958f (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:30.032879 14869 raft_consensus.cc:2932] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:30.033308 14721 catalog_manager.cc:5484] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 reported cstate change: config changed from index 4 to 5, 58593562dc504df98fbba7f0b734958f (127.0.99.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:30.033535 15723 raft_consensus.cc:2932] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:30.035883 14726 catalog_manager.cc:5484] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff reported cstate change: config changed from index 4 to 5, 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:30.038514 14870 raft_consensus.cc:2932] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 58593562dc504df98fbba7f0b734958f (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } attrs { replace: true } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:30.038741 15266 consensus_queue.cc:237] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } }
I20240501 08:00:30.041306 15592 raft_consensus.cc:1261] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:30.041618 15266 consensus_queue.cc:237] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } }
I20240501 08:00:30.041769 16186 consensus_queue.cc:1035] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:30.042851 15723 raft_consensus.cc:1261] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 FOLLOWER]: Refusing update from remote peer e019f42a04ca40b491694f86026e3638: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:30.043248 16186 consensus_queue.cc:1035] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:30.043428 16190 raft_consensus.cc:2932] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6016871632a84df2abdcc5690f07cb9e (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:30.044132 15592 raft_consensus.cc:2932] T 599cc1bd079449209998e39e5cfee2a5 P 58593562dc504df98fbba7f0b734958f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6016871632a84df2abdcc5690f07cb9e (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:30.045107 14715 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 599cc1bd079449209998e39e5cfee2a5 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:30.045867 14726 catalog_manager.cc:5484] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 reported cstate change: config changed from index 5 to 6, VOTER 6016871632a84df2abdcc5690f07cb9e (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:30.047850 16186 raft_consensus.cc:2932] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6016871632a84df2abdcc5690f07cb9e (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:30.049150 14850 tablet_service.cc:1514] Processing DeleteTablet for tablet 599cc1bd079449209998e39e5cfee2a5 with delete_type TABLET_DATA_TOMBSTONED (TS 6016871632a84df2abdcc5690f07cb9e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38758
I20240501 08:00:30.049377 16185 tablet_replica.cc:330] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e: stopping tablet replica
I20240501 08:00:30.049422 14715 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f809a3f2420f4e8398cd05529886691b with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:30.049522 16185 raft_consensus.cc:2229] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:30.049686 16185 raft_consensus.cc:2258] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:30.049635 15723 raft_consensus.cc:2932] T f809a3f2420f4e8398cd05529886691b P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 6016871632a84df2abdcc5690f07cb9e (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:30.051653 14726 catalog_manager.cc:5484] T f809a3f2420f4e8398cd05529886691b P e019f42a04ca40b491694f86026e3638 reported cstate change: config changed from index 5 to 6, VOTER 6016871632a84df2abdcc5690f07cb9e (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "e019f42a04ca40b491694f86026e3638" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:30.053786 16185 ts_tablet_manager.cc:1875] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:30.055089 14850 tablet_service.cc:1514] Processing DeleteTablet for tablet f809a3f2420f4e8398cd05529886691b with delete_type TABLET_DATA_TOMBSTONED (TS 6016871632a84df2abdcc5690f07cb9e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38758
I20240501 08:00:30.056017 16185 ts_tablet_manager.cc:1888] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:30.056111 16185 log.cc:1196] T 599cc1bd079449209998e39e5cfee2a5 P 6016871632a84df2abdcc5690f07cb9e: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/599cc1bd079449209998e39e5cfee2a5
I20240501 08:00:30.056540 16185 tablet_replica.cc:330] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e: stopping tablet replica
I20240501 08:00:30.056597 14713 catalog_manager.cc:4830] TS 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697): tablet 599cc1bd079449209998e39e5cfee2a5 (table rebalance_test_table_2 [id=58dc1c9d84704008a42cea20932bda35]) successfully deleted
I20240501 08:00:30.056672 16185 raft_consensus.cc:2229] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:30.056821 16185 raft_consensus.cc:2258] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:30.057412 16185 ts_tablet_manager.cc:1875] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:30.059295 16185 ts_tablet_manager.cc:1888] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:30.059357 16185 log.cc:1196] T f809a3f2420f4e8398cd05529886691b P 6016871632a84df2abdcc5690f07cb9e: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/f809a3f2420f4e8398cd05529886691b
I20240501 08:00:30.059788 14713 catalog_manager.cc:4830] TS 6016871632a84df2abdcc5690f07cb9e (127.0.99.129:39697): tablet f809a3f2420f4e8398cd05529886691b (table rebalance_test_table_1 [id=b74463b6b30c4a46989fc21f0cb1ec68]) successfully deleted
I20240501 08:00:30.291990 16069 raft_consensus.cc:1050] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e: attempting to promote NON_VOTER 7a8a3e5fb2c14eb9986d9da6cb1229ff to VOTER
I20240501 08:00:30.292436 16069 consensus_queue.cc:237] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } }
I20240501 08:00:30.293879 15723 raft_consensus.cc:1261] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 LEARNER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:30.293879 15266 raft_consensus.cc:1261] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:30.294307 15924 consensus_queue.cc:1035] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:30.294553 16111 consensus_queue.cc:1035] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:30.296860 16069 raft_consensus.cc:2932] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:30.298712 15266 raft_consensus.cc:2932] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:30.298748 15723 raft_consensus.cc:2932] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:30.299474 14727 catalog_manager.cc:5484] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: config changed from index 4 to 5, 7a8a3e5fb2c14eb9986d9da6cb1229ff (127.0.99.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:30.302624 14870 consensus_queue.cc:237] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } }
I20240501 08:00:30.303815 15723 raft_consensus.cc:1261] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 FOLLOWER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:30.304417 16069 consensus_queue.cc:1035] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:30.306351 15924 raft_consensus.cc:2932] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e019f42a04ca40b491694f86026e3638 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:30.306898 15723 raft_consensus.cc:2932] T 1d19a2efa6194a1b9321b7b15b84afb1 P 7a8a3e5fb2c14eb9986d9da6cb1229ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e019f42a04ca40b491694f86026e3638 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } } }
I20240501 08:00:30.307760 14713 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1d19a2efa6194a1b9321b7b15b84afb1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:30.308758 14724 catalog_manager.cc:5484] T 1d19a2efa6194a1b9321b7b15b84afb1 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: config changed from index 5 to 6, VOTER e019f42a04ca40b491694f86026e3638 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a8a3e5fb2c14eb9986d9da6cb1229ff" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34069 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:30.311512 15245 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d19a2efa6194a1b9321b7b15b84afb1 with delete_type TABLET_DATA_TOMBSTONED (TS e019f42a04ca40b491694f86026e3638 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:51886
I20240501 08:00:30.311753 16184 tablet_replica.cc:330] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638: stopping tablet replica
I20240501 08:00:30.311880 16184 raft_consensus.cc:2229] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:30.312077 16184 raft_consensus.cc:2258] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:30.313405 16184 ts_tablet_manager.cc:1875] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:30.316149 16184 ts_tablet_manager.cc:1888] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:30.316246 16184 log.cc:1196] T 1d19a2efa6194a1b9321b7b15b84afb1 P e019f42a04ca40b491694f86026e3638: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/1d19a2efa6194a1b9321b7b15b84afb1
I20240501 08:00:30.316785 14715 catalog_manager.cc:4830] TS e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573): tablet 1d19a2efa6194a1b9321b7b15b84afb1 (table rebalance_test_table_2 [id=58dc1c9d84704008a42cea20932bda35]) successfully deleted
I20240501 08:00:30.328536 15924 raft_consensus.cc:1050] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e: attempting to promote NON_VOTER 58593562dc504df98fbba7f0b734958f to VOTER
I20240501 08:00:30.328971 15924 consensus_queue.cc:237] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } }
I20240501 08:00:30.330394 15592 raft_consensus.cc:1261] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f [term 1 LEARNER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:30.330394 15266 raft_consensus.cc:1261] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:30.330830 16069 consensus_queue.cc:1035] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:30.331349 16069 consensus_queue.cc:1035] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:30.333046 16069 raft_consensus.cc:2932] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 58593562dc504df98fbba7f0b734958f (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:30.333541 15266 raft_consensus.cc:2932] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 58593562dc504df98fbba7f0b734958f (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:30.335606 15592 raft_consensus.cc:2932] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 58593562dc504df98fbba7f0b734958f (127.0.99.130) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:30.336030 14726 catalog_manager.cc:5484] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: config changed from index 4 to 5, 58593562dc504df98fbba7f0b734958f (127.0.99.130) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e019f42a04ca40b491694f86026e3638" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:30.339470 14870 consensus_queue.cc:237] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } }
I20240501 08:00:30.340642 15592 raft_consensus.cc:1261] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f [term 1 FOLLOWER]: Refusing update from remote peer 6016871632a84df2abdcc5690f07cb9e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:30.341022 16271 consensus_queue.cc:1035] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:30.342715 15924 raft_consensus.cc:2932] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e019f42a04ca40b491694f86026e3638 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:30.343150 15592 raft_consensus.cc:2932] T aaebe02e13964ab88e1f6c24a9ce8131 P 58593562dc504df98fbba7f0b734958f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e019f42a04ca40b491694f86026e3638 (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } } }
I20240501 08:00:30.344211 14713 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet aaebe02e13964ab88e1f6c24a9ce8131 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:30.345470 14727 catalog_manager.cc:5484] T aaebe02e13964ab88e1f6c24a9ce8131 P 6016871632a84df2abdcc5690f07cb9e reported cstate change: config changed from index 5 to 6, VOTER e019f42a04ca40b491694f86026e3638 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "6016871632a84df2abdcc5690f07cb9e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6016871632a84df2abdcc5690f07cb9e" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58593562dc504df98fbba7f0b734958f" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41275 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:30.348114 15245 tablet_service.cc:1514] Processing DeleteTablet for tablet aaebe02e13964ab88e1f6c24a9ce8131 with delete_type TABLET_DATA_TOMBSTONED (TS e019f42a04ca40b491694f86026e3638 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:51886
I20240501 08:00:30.348367 16184 tablet_replica.cc:330] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638: stopping tablet replica
I20240501 08:00:30.348517 16184 raft_consensus.cc:2229] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:30.348695 16184 raft_consensus.cc:2258] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:30.349365 16184 ts_tablet_manager.cc:1875] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:30.351203 16184 ts_tablet_manager.cc:1888] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 08:00:30.351282 16184 log.cc:1196] T aaebe02e13964ab88e1f6c24a9ce8131 P e019f42a04ca40b491694f86026e3638: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/aaebe02e13964ab88e1f6c24a9ce8131
I20240501 08:00:30.351866 14715 catalog_manager.cc:4830] TS e019f42a04ca40b491694f86026e3638 (127.0.99.132:40573): tablet aaebe02e13964ab88e1f6c24a9ce8131 (table rebalance_test_table_1 [id=b74463b6b30c4a46989fc21f0cb1ec68]) successfully deleted
I20240501 08:00:30.594107 14850 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:30.595332 15703 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:30.595587 15572 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:30.604208 15245 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:30.693913 15572 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:30.697917 14850 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:30.701895 15703 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:30.709242 15245 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:30.789834 15703 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:30.789937 15572 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:30.790897 14850 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:30.802716 15245 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:30.833514 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 14788
I20240501 08:00:30.840134 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 15510
I20240501 08:00:30.846060 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 15641
W20240501 08:00:30.848482 15201 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.99.130:41275: connect: Connection refused (error 111)
W20240501 08:00:30.849673 15201 consensus_peers.cc:485] T 599cc1bd079449209998e39e5cfee2a5 P e019f42a04ca40b491694f86026e3638 -> Peer 58593562dc504df98fbba7f0b734958f (127.0.99.130:41275): Couldn't send request to peer 58593562dc504df98fbba7f0b734958f. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.130:41275: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:30.852252 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 15183
I20240501 08:00:30.858441 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 14697
2024-05-01T08:00:30Z chronyd exiting
[ OK ] LocationAwareRebalancingParamTest.Rebalance/idx7_rf2_t3_l_1_2_1 (10118 ms)
[ RUN ] LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer/idx6_rf7_t7_l_3_4_3
I20240501 08:00:30.883023 398 test_util.cc:268] Using random seed: -513680187
I20240501 08:00:30.883906 398 ts_itest-base.cc:112] Starting cluster with:
I20240501 08:00:30.884382 398 ts_itest-base.cc:113] --------------
I20240501 08:00:30.884404 398 ts_itest-base.cc:114] 10 tablet servers
I20240501 08:00:30.884419 398 ts_itest-base.cc:115] 7 replicas per TS
I20240501 08:00:30.884431 398 ts_itest-base.cc:116] --------------
2024-05-01T08:00:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T08:00:30Z Disabled control of system clock
I20240501 08:00:30.894834 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:35687
--webserver_interface=127.0.99.190
--webserver_port=0
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.99.190:35687
--location_mapping_cmd=/tmp/dist-test-taskrYyCEf/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/location-assignment.state --map /A:3 --map /B:4 --map /C:3
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false
--master_client_location_assignment_enabled=false
--allow_unsafe_replication_factor with env {}
W20240501 08:00:31.008641 16390 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240501 08:00:31.008945 16390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:31.008971 16390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:31.009009 16390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:31.012388 16390 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:31.012482 16390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 08:00:31.012509 16390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:31.012526 16390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 08:00:31.012542 16390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 08:00:31.016583 16390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--location_mapping_cmd=/tmp/dist-test-taskrYyCEf/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/location-assignment.state --map /A:3 --map /B:4 --map /C:3
--ipki_ca_key_size=768
--master_addresses=127.0.99.190:35687
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:35687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.99.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.16390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:31.017589 16390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:31.018777 16390 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:31.027619 16398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:31.028395 16396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:31.029214 16395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:31.029894 16390 server_base.cc:1034] running on GCE node
I20240501 08:00:31.030493 16390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:31.031494 16390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:31.032666 16390 hybrid_clock.cc:648] HybridClock initialized: now 1714550431032645 us; error 42 us; skew 500 ppm
I20240501 08:00:31.034870 16390 webserver.cc:460] Webserver started at http://127.0.99.190:33533/ using document root <none> and password file <none>
I20240501 08:00:31.035504 16390 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:31.035573 16390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:31.035914 16390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:31.037915 16390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "bb50a0e0131f484db8236d7bd396d576"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.038458 16390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "bb50a0e0131f484db8236d7bd396d576"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.042310 16390 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20240501 08:00:31.044850 16404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:31.046065 16390 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20240501 08:00:31.046231 16390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
uuid: "bb50a0e0131f484db8236d7bd396d576"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.046347 16390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:31.086833 16390 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:31.087533 16390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:31.087710 16390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:31.094517 16456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:35687 every 8 connection(s)
I20240501 08:00:31.094518 16390 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:35687
I20240501 08:00:31.095724 16390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 08:00:31.098662 16457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:31.101898 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 16390
I20240501 08:00:31.102043 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 08:00:31.104507 16457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576: Bootstrap starting.
I20240501 08:00:31.107268 16457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:31.108158 16457 log.cc:824] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:31.110253 16457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576: No bootstrap required, opened a new log
I20240501 08:00:31.113451 16457 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb50a0e0131f484db8236d7bd396d576" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 35687 } }
I20240501 08:00:31.113840 16457 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:31.113941 16457 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb50a0e0131f484db8236d7bd396d576, State: Initialized, Role: FOLLOWER
I20240501 08:00:31.114384 16457 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb50a0e0131f484db8236d7bd396d576" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 35687 } }
I20240501 08:00:31.114518 16457 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 08:00:31.114591 16457 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 08:00:31.114702 16457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:31.116031 16457 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb50a0e0131f484db8236d7bd396d576" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 35687 } }
I20240501 08:00:31.116410 16457 leader_election.cc:302] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb50a0e0131f484db8236d7bd396d576; no voters:
I20240501 08:00:31.116727 16457 leader_election.cc:288] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 08:00:31.116883 16462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:31.117098 16462 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [term 1 LEADER]: Becoming Leader. State: Replica: bb50a0e0131f484db8236d7bd396d576, State: Running, Role: LEADER
I20240501 08:00:31.117413 16462 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb50a0e0131f484db8236d7bd396d576" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 35687 } }
I20240501 08:00:31.117897 16457 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 08:00:31.119031 16464 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb50a0e0131f484db8236d7bd396d576. Latest consensus state: current_term: 1 leader_uuid: "bb50a0e0131f484db8236d7bd396d576" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb50a0e0131f484db8236d7bd396d576" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 35687 } } }
I20240501 08:00:31.119210 16464 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [sys.catalog]: This master's current role is: LEADER
I20240501 08:00:31.119575 16471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 08:00:31.120199 16463 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb50a0e0131f484db8236d7bd396d576" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb50a0e0131f484db8236d7bd396d576" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 35687 } } }
I20240501 08:00:31.120308 16463 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576 [sys.catalog]: This master's current role is: LEADER
I20240501 08:00:31.123335 16471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 08:00:31.130012 16471 catalog_manager.cc:1347] Generated new cluster ID: 63714d3fc9a346cba283de8f19cf8039
I20240501 08:00:31.130113 16471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 08:00:31.153438 16471 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 08:00:31.154570 16471 catalog_manager.cc:1504] Loading token signing keys...
I20240501 08:00:31.161998 16471 catalog_manager.cc:5857] T 00000000000000000000000000000000 P bb50a0e0131f484db8236d7bd396d576: Generated new TSK 0
I20240501 08:00:31.162825 16471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 08:00:31.172291 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.129:0
--local_ip_for_outbound_sockets=127.0.99.129
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 08:00:31.331276 16481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:31.331597 16481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:31.331662 16481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:31.336341 16481 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:31.336452 16481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:31.336555 16481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.129
I20240501 08:00:31.342056 16481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.16481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.129
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:31.343218 16481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:31.344761 16481 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:31.354025 16489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:31.354470 16487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:31.355374 16486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:31.356351 16481 server_base.cc:1034] running on GCE node
I20240501 08:00:31.356848 16481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:31.357501 16481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:31.358690 16481 hybrid_clock.cc:648] HybridClock initialized: now 1714550431358657 us; error 52 us; skew 500 ppm
I20240501 08:00:31.365530 16481 webserver.cc:460] Webserver started at http://127.0.99.129:38629/ using document root <none> and password file <none>
I20240501 08:00:31.366180 16481 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:31.366250 16481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:31.366441 16481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:31.368292 16481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "eca6f08d57d1453784f726199474a671"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.368810 16481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "eca6f08d57d1453784f726199474a671"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.372506 16481 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:31.375236 16495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:31.376873 16481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:31.377035 16481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "eca6f08d57d1453784f726199474a671"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.377156 16481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:31.394958 16481 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:31.395690 16481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:31.395870 16481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:31.396523 16481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:31.397501 16481 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:31.397552 16481 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:31.397634 16481 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:31.397660 16481 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:31.406459 16481 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:43741
I20240501 08:00:31.406527 16608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:43741 every 8 connection(s)
I20240501 08:00:31.407523 16481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 08:00:31.411026 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 16481
I20240501 08:00:31.411171 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 08:00:31.413586 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:0
--local_ip_for_outbound_sockets=127.0.99.130
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:31.418360 16609 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:31.418642 16609 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:31.419317 16609 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:31.473174 16421 ts_manager.cc:194] Registered new tserver with Master: eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:00:31.474561 16421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.129:51885
W20240501 08:00:31.548897 16612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:31.549177 16612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:31.549242 16612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:31.552702 16612 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:31.552805 16612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:31.552897 16612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240501 08:00:31.557094 16612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.16612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:31.558148 16612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:31.559370 16612 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:31.569007 16618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:31.569741 16621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:31.570622 16619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:31.571188 16612 server_base.cc:1034] running on GCE node
I20240501 08:00:31.571695 16612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:31.572422 16612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:31.573583 16612 hybrid_clock.cc:648] HybridClock initialized: now 1714550431573576 us; error 30 us; skew 500 ppm
I20240501 08:00:31.575781 16612 webserver.cc:460] Webserver started at http://127.0.99.130:36623/ using document root <none> and password file <none>
I20240501 08:00:31.576457 16612 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:31.576526 16612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:31.576714 16612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:31.578593 16612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "64196487691e4623b690b2869aad57f7"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.579159 16612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "64196487691e4623b690b2869aad57f7"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.583304 16612 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20240501 08:00:31.585994 16627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:31.587258 16612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240501 08:00:31.587440 16612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "64196487691e4623b690b2869aad57f7"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.587555 16612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:31.603281 16612 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:31.604033 16612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:31.604200 16612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:31.604748 16612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:31.605703 16612 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:31.605752 16612 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:31.605809 16612 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:31.605839 16612 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:31.614650 16612 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:45219
I20240501 08:00:31.614715 16740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:45219 every 8 connection(s)
I20240501 08:00:31.615690 16612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 08:00:31.620944 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 16612
I20240501 08:00:31.621075 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 08:00:31.623262 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:0
--local_ip_for_outbound_sockets=127.0.99.131
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:31.626004 16741 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:31.626355 16741 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:31.627241 16741 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:31.677704 16421 ts_manager.cc:194] Registered new tserver with Master: 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219)
I20240501 08:00:31.678470 16421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:42585
W20240501 08:00:31.757098 16744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:31.757351 16744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:31.757403 16744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:31.760780 16744 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:31.760870 16744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:31.760946 16744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240501 08:00:31.764953 16744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.16744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:31.766038 16744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:31.767158 16744 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:31.776731 16753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:31.777829 16750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:31.779579 16751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:31.779704 16744 server_base.cc:1034] running on GCE node
I20240501 08:00:31.780256 16744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:31.780984 16744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:31.782160 16744 hybrid_clock.cc:648] HybridClock initialized: now 1714550431782135 us; error 46 us; skew 500 ppm
I20240501 08:00:31.784693 16744 webserver.cc:460] Webserver started at http://127.0.99.131:43159/ using document root <none> and password file <none>
I20240501 08:00:31.785349 16744 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:31.785419 16744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:31.785609 16744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:31.787478 16744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "1160c6e72055456992b5d8bf5f3b45c0"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.788050 16744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "1160c6e72055456992b5d8bf5f3b45c0"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.792433 16744 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20240501 08:00:31.795367 16759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:31.796778 16744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:31.796962 16744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "1160c6e72055456992b5d8bf5f3b45c0"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.797082 16744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:31.823359 16744 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:31.824075 16744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:31.824246 16744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:31.824820 16744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:31.825800 16744 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:31.825850 16744 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:31.825906 16744 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:31.825932 16744 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:31.834723 16744 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:43779
I20240501 08:00:31.834751 16872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:43779 every 8 connection(s)
I20240501 08:00:31.835774 16744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 08:00:31.840978 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 16744
I20240501 08:00:31.841219 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 08:00:31.844113 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:0
--local_ip_for_outbound_sockets=127.0.99.132
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:31.846479 16873 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:31.846772 16873 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:31.847527 16873 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:31.888294 16421 ts_manager.cc:194] Registered new tserver with Master: 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:31.889283 16421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:36649
W20240501 08:00:31.961589 16876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:31.961844 16876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:31.961898 16876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:31.965217 16876 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:31.965312 16876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:31.965385 16876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240501 08:00:31.969408 16876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.16876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:31.970405 16876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:31.971577 16876 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:31.981797 16882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:31.982453 16883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:31.982985 16885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:31.984133 16876 server_base.cc:1034] running on GCE node
I20240501 08:00:31.984652 16876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:31.985339 16876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:31.986554 16876 hybrid_clock.cc:648] HybridClock initialized: now 1714550431986526 us; error 43 us; skew 500 ppm
I20240501 08:00:31.988996 16876 webserver.cc:460] Webserver started at http://127.0.99.132:41985/ using document root <none> and password file <none>
I20240501 08:00:31.989626 16876 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:31.989693 16876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:31.989941 16876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:31.992024 16876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.992623 16876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:31.997196 16876 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20240501 08:00:32.000186 16891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.001632 16876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20240501 08:00:32.001821 16876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
format_stamp: "Formatted at 2024-05-01 08:00:31 on dist-test-slave-f81v"
I20240501 08:00:32.001937 16876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:32.019276 16876 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:32.020056 16876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:32.020216 16876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:32.020809 16876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:32.021791 16876 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:32.021847 16876 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.021893 16876 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:32.021920 16876 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.030596 16876 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:41987
I20240501 08:00:32.030630 17004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:41987 every 8 connection(s)
I20240501 08:00:32.031662 16876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 08:00:32.041196 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 16876
I20240501 08:00:32.041339 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 08:00:32.042132 17005 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:32.042397 17005 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:32.043027 17005 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:32.043514 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:0
--local_ip_for_outbound_sockets=127.0.99.133
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:32.081055 16421 ts_manager.cc:194] Registered new tserver with Master: d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987)
I20240501 08:00:32.082103 16421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:52081
W20240501 08:00:32.166983 17008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:32.167238 17008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:32.167285 17008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:32.170697 17008 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:32.170790 17008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:32.170862 17008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240501 08:00:32.174901 17008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.17008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:32.175884 17008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:32.177134 17008 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:32.187288 17014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:32.187834 17017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:32.188813 17015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:32.189541 17008 server_base.cc:1034] running on GCE node
I20240501 08:00:32.190099 17008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:32.190752 17008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:32.191983 17008 hybrid_clock.cc:648] HybridClock initialized: now 1714550432191882 us; error 53 us; skew 500 ppm
I20240501 08:00:32.194355 17008 webserver.cc:460] Webserver started at http://127.0.99.133:45687/ using document root <none> and password file <none>
I20240501 08:00:32.195001 17008 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:32.195067 17008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:32.195261 17008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:32.197185 17008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "e282199dc2fa4977b83ff07b86cb0690"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:32.197754 17008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "e282199dc2fa4977b83ff07b86cb0690"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:32.201807 17008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20240501 08:00:32.204720 17023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.206032 17008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:32.206187 17008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "e282199dc2fa4977b83ff07b86cb0690"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:32.206310 17008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:32.219579 17008 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:32.220376 17008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:32.220546 17008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:32.221130 17008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:32.222127 17008 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:32.222177 17008 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.222219 17008 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:32.222239 17008 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.231456 17008 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:41463
I20240501 08:00:32.231491 17136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:41463 every 8 connection(s)
I20240501 08:00:32.232578 17008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 08:00:32.241286 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 17008
I20240501 08:00:32.241468 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240501 08:00:32.244103 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.134:0
--local_ip_for_outbound_sockets=127.0.99.134
--webserver_interface=127.0.99.134
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:32.244396 17137 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:32.244676 17137 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:32.245352 17137 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:32.294628 16421 ts_manager.cc:194] Registered new tserver with Master: e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463)
I20240501 08:00:32.295394 16421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:59415
W20240501 08:00:32.374927 17140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:32.375214 17140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:32.375263 17140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:32.378943 17140 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:32.379040 17140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:32.379120 17140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.134
I20240501 08:00:32.383381 17140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.0.99.134
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.17140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.134
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:32.384531 17140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:32.385829 17140 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:32.396137 17149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:32.396720 17147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:32.397545 17146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:32.398192 17140 server_base.cc:1034] running on GCE node
I20240501 08:00:32.398645 17140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:32.399295 17140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:32.400465 17140 hybrid_clock.cc:648] HybridClock initialized: now 1714550432400430 us; error 53 us; skew 500 ppm
I20240501 08:00:32.402911 17140 webserver.cc:460] Webserver started at http://127.0.99.134:43847/ using document root <none> and password file <none>
I20240501 08:00:32.403525 17140 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:32.403582 17140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:32.403753 17140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:32.405656 17140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "3d06b1f1890443da898fde2eb82daad3"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:32.406257 17140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "3d06b1f1890443da898fde2eb82daad3"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:32.410282 17140 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20240501 08:00:32.413026 17155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.414268 17140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 08:00:32.414429 17140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "3d06b1f1890443da898fde2eb82daad3"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:32.414537 17140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:32.441102 17140 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:32.441850 17140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:32.442042 17140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:32.442641 17140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:32.443637 17140 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:32.443689 17140 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.443745 17140 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:32.443773 17140 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.453855 17140 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.134:46813
I20240501 08:00:32.453884 17268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.134:46813 every 8 connection(s)
I20240501 08:00:32.454905 17140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240501 08:00:32.461477 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 17140
I20240501 08:00:32.461625 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240501 08:00:32.464171 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.135:0
--local_ip_for_outbound_sockets=127.0.99.135
--webserver_interface=127.0.99.135
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:32.468500 17269 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:32.468801 17269 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:32.469429 17269 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:32.477492 16609 heartbeater.cc:503] Master 127.0.99.190:35687 was elected leader, sending a full tablet report...
I20240501 08:00:32.514153 16421 ts_manager.cc:194] Registered new tserver with Master: 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813)
I20240501 08:00:32.514870 16421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.134:39845
W20240501 08:00:32.592033 17272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:32.592310 17272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:32.592383 17272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:32.595811 17272 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:32.595913 17272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:32.596037 17272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.135
I20240501 08:00:32.600072 17272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.0.99.135
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.17272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.135
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:32.601104 17272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:32.602360 17272 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:32.611819 17281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:32.612211 17279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:32.613147 17278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:32.613883 17272 server_base.cc:1034] running on GCE node
I20240501 08:00:32.614329 17272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:32.615090 17272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:32.616293 17272 hybrid_clock.cc:648] HybridClock initialized: now 1714550432616267 us; error 43 us; skew 500 ppm
I20240501 08:00:32.619040 17272 webserver.cc:460] Webserver started at http://127.0.99.135:38857/ using document root <none> and password file <none>
I20240501 08:00:32.619642 17272 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:32.619714 17272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:32.619899 17272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:32.621727 17272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "af70005d0c6c48bc9919cae6346f221b"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:32.622225 17272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "af70005d0c6c48bc9919cae6346f221b"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:32.625905 17272 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240501 08:00:32.628386 17287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.629606 17272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 08:00:32.629741 17272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "af70005d0c6c48bc9919cae6346f221b"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:32.629851 17272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:32.642962 17272 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:32.643656 17272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:32.643826 17272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:32.644424 17272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:32.645371 17272 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:32.645421 17272 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.645466 17272 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:32.645504 17272 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.655086 17272 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.135:39065
I20240501 08:00:32.655159 17400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.135:39065 every 8 connection(s)
I20240501 08:00:32.656162 17272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 08:00:32.661088 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 17272
I20240501 08:00:32.661208 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240501 08:00:32.663479 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.136:0
--local_ip_for_outbound_sockets=127.0.99.136
--webserver_interface=127.0.99.136
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:32.666229 17401 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:32.666520 17401 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:32.667135 17401 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:32.681455 16741 heartbeater.cc:503] Master 127.0.99.190:35687 was elected leader, sending a full tablet report...
I20240501 08:00:32.727368 16421 ts_manager.cc:194] Registered new tserver with Master: af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:32.728413 16421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.135:56275
W20240501 08:00:32.781944 17404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:32.782222 17404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:32.782272 17404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:32.785773 17404 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:32.785858 17404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:32.785951 17404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.136
I20240501 08:00:32.790057 17404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.136:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.0.99.136
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.17404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.136
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:32.791101 17404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:32.792389 17404 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:32.802234 17411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:32.802656 17413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:32.803471 17410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:32.804345 17404 server_base.cc:1034] running on GCE node
I20240501 08:00:32.804864 17404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:32.805521 17404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:32.806706 17404 hybrid_clock.cc:648] HybridClock initialized: now 1714550432806693 us; error 37 us; skew 500 ppm
I20240501 08:00:32.809049 17404 webserver.cc:460] Webserver started at http://127.0.99.136:36335/ using document root <none> and password file <none>
I20240501 08:00:32.809660 17404 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:32.809727 17404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:32.809927 17404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:32.811805 17404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "5aaf071b45ab469cb23321d37fad37f8"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:32.812373 17404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "5aaf071b45ab469cb23321d37fad37f8"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:32.815996 17404 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240501 08:00:32.818585 17419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.819878 17404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20240501 08:00:32.820067 17404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "5aaf071b45ab469cb23321d37fad37f8"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:32.820181 17404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:32.848618 17404 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:32.849309 17404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:32.849478 17404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:32.850044 17404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:32.850977 17404 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:32.851032 17404 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.851076 17404 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:32.851105 17404 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:32.859658 17404 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.136:34371
I20240501 08:00:32.859688 17532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.136:34371 every 8 connection(s)
I20240501 08:00:32.860723 17404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 08:00:32.870124 17533 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:32.870380 17533 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:32.870383 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 17404
I20240501 08:00:32.870527 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240501 08:00:32.871006 17533 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:32.872861 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.137:0
--local_ip_for_outbound_sockets=127.0.99.137
--webserver_interface=127.0.99.137
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:32.892379 16873 heartbeater.cc:503] Master 127.0.99.190:35687 was elected leader, sending a full tablet report...
I20240501 08:00:32.908116 16421 ts_manager.cc:194] Registered new tserver with Master: 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371)
I20240501 08:00:32.908867 16421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.136:38999
W20240501 08:00:32.988373 17537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:32.988626 17537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:32.988674 17537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:32.992017 17537 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:32.992105 17537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:32.992192 17537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.137
I20240501 08:00:32.996253 17537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.137:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.0.99.137
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.17537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.137
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:32.997246 17537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:32.998440 17537 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:33.007643 17545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:33.008234 17542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:33.009042 17543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:33.009959 17537 server_base.cc:1034] running on GCE node
I20240501 08:00:33.010450 17537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:33.011072 17537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:33.012245 17537 hybrid_clock.cc:648] HybridClock initialized: now 1714550433012231 us; error 30 us; skew 500 ppm
I20240501 08:00:33.014477 17537 webserver.cc:460] Webserver started at http://127.0.99.137:46789/ using document root <none> and password file <none>
I20240501 08:00:33.015089 17537 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:33.015151 17537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:33.015389 17537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:33.017282 17537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/instance:
uuid: "873218458f2346348be2120e543990e2"
format_stamp: "Formatted at 2024-05-01 08:00:33 on dist-test-slave-f81v"
I20240501 08:00:33.017989 17537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal/instance:
uuid: "873218458f2346348be2120e543990e2"
format_stamp: "Formatted at 2024-05-01 08:00:33 on dist-test-slave-f81v"
I20240501 08:00:33.022274 17537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20240501 08:00:33.024928 17551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:33.026090 17537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 08:00:33.026224 17537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "873218458f2346348be2120e543990e2"
format_stamp: "Formatted at 2024-05-01 08:00:33 on dist-test-slave-f81v"
I20240501 08:00:33.026335 17537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:33.046757 17537 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:33.047511 17537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:33.047690 17537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:33.048346 17537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:33.049391 17537 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:33.049441 17537 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:33.049486 17537 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:33.049515 17537 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:33.058341 17537 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.137:33839
I20240501 08:00:33.058377 17664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.137:33839 every 8 connection(s)
I20240501 08:00:33.059386 17537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240501 08:00:33.069053 17665 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:33.069329 17665 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:33.069474 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 17537
I20240501 08:00:33.069604 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal/instance
I20240501 08:00:33.070102 17665 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:33.071807 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.138:0
--local_ip_for_outbound_sockets=127.0.99.138
--webserver_interface=127.0.99.138
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:33.084820 17005 heartbeater.cc:503] Master 127.0.99.190:35687 was elected leader, sending a full tablet report...
I20240501 08:00:33.112120 16421 ts_manager.cc:194] Registered new tserver with Master: 873218458f2346348be2120e543990e2 (127.0.99.137:33839)
I20240501 08:00:33.113006 16421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.137:48495
W20240501 08:00:33.200419 17669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:33.200673 17669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:33.200721 17669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:33.204330 17669 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:33.204413 17669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:33.204493 17669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.138
I20240501 08:00:33.208662 17669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.138:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--webserver_interface=127.0.99.138
--webserver_port=0
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.17669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.138
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:33.209750 17669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:33.210990 17669 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:33.220619 17675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:33.220957 17674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:33.221451 17677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:33.222334 17669 server_base.cc:1034] running on GCE node
I20240501 08:00:33.222790 17669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:33.223803 17669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:33.225035 17669 hybrid_clock.cc:648] HybridClock initialized: now 1714550433225015 us; error 38 us; skew 500 ppm
I20240501 08:00:33.227275 17669 webserver.cc:460] Webserver started at http://127.0.99.138:39607/ using document root <none> and password file <none>
I20240501 08:00:33.227900 17669 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:33.228014 17669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:33.228211 17669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 08:00:33.230023 17669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data/instance:
uuid: "5f8132e5a8014a6f913ee9ac3338600a"
format_stamp: "Formatted at 2024-05-01 08:00:33 on dist-test-slave-f81v"
I20240501 08:00:33.230592 17669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/wal/instance:
uuid: "5f8132e5a8014a6f913ee9ac3338600a"
format_stamp: "Formatted at 2024-05-01 08:00:33 on dist-test-slave-f81v"
I20240501 08:00:33.235338 17669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20240501 08:00:33.238075 17683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:33.239362 17669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:33.239509 17669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/wal
uuid: "5f8132e5a8014a6f913ee9ac3338600a"
format_stamp: "Formatted at 2024-05-01 08:00:33 on dist-test-slave-f81v"
I20240501 08:00:33.239624 17669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:33.263865 17669 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:33.264616 17669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:33.264781 17669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:33.265342 17669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:33.266285 17669 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:33.266333 17669 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:33.266386 17669 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:33.266414 17669 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:33.274821 17669 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.138:37529
I20240501 08:00:33.274847 17796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.138:37529 every 8 connection(s)
I20240501 08:00:33.275805 17669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data/info.pb
I20240501 08:00:33.278570 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 17669
I20240501 08:00:33.278756 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/wal/instance
I20240501 08:00:33.285763 17797 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:33.286028 17797 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:33.286712 17797 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:33.297814 17137 heartbeater.cc:503] Master 127.0.99.190:35687 was elected leader, sending a full tablet report...
I20240501 08:00:33.324116 16421 ts_manager.cc:194] Registered new tserver with Master: 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529)
I20240501 08:00:33.324962 16421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.138:44511
I20240501 08:00:33.332028 398 external_mini_cluster.cc:930] 10 TS(s) registered with all masters
I20240501 08:00:33.344621 398 external_mini_cluster.cc:930] 10 TS(s) registered with all masters
I20240501 08:00:33.344730 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 17404
I20240501 08:00:33.349793 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 17537
I20240501 08:00:33.355010 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu with pid 17669
I20240501 08:00:33.517446 17269 heartbeater.cc:503] Master 127.0.99.190:35687 was elected leader, sending a full tablet report...
I20240501 08:00:33.731103 17401 heartbeater.cc:503] Master 127.0.99.190:35687 was elected leader, sending a full tablet report...
I20240501 08:00:37.111970 16421 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59002:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 08:00:37.116622 16421 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 08:00:37.139667 17334 tablet_service.cc:1467] Processing CreateTablet for tablet 0d27d6c2cc194773a648909806f4416b (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.139801 17070 tablet_service.cc:1467] Processing CreateTablet for tablet 0d27d6c2cc194773a648909806f4416b (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.139801 17071 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8a14c2447e45c5b148ff762bb55c00 (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.141074 17334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d27d6c2cc194773a648909806f4416b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.142036 17071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8a14c2447e45c5b148ff762bb55c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.141803 17069 tablet_service.cc:1467] Processing CreateTablet for tablet 452b5685e6ca4793a6628d700599bdbb (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.142982 17070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d27d6c2cc194773a648909806f4416b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.143586 17069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452b5685e6ca4793a6628d700599bdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.144351 17333 tablet_service.cc:1467] Processing CreateTablet for tablet 452b5685e6ca4793a6628d700599bdbb (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.144680 17333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452b5685e6ca4793a6628d700599bdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.150323 17335 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8a14c2447e45c5b148ff762bb55c00 (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.150779 17335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8a14c2447e45c5b148ff762bb55c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.151217 17203 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8a14c2447e45c5b148ff762bb55c00 (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.151434 16937 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8a14c2447e45c5b148ff762bb55c00 (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.152278 17202 tablet_service.cc:1467] Processing CreateTablet for tablet 0d27d6c2cc194773a648909806f4416b (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.152390 16937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8a14c2447e45c5b148ff762bb55c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.152689 17201 tablet_service.cc:1467] Processing CreateTablet for tablet 452b5685e6ca4793a6628d700599bdbb (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.153272 17203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8a14c2447e45c5b148ff762bb55c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.155460 17824 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:37.155966 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452b5685e6ca4793a6628d700599bdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.156153 16939 tablet_service.cc:1467] Processing CreateTablet for tablet 0d27d6c2cc194773a648909806f4416b (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.156446 16939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d27d6c2cc194773a648909806f4416b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.158170 17824 tablet_bootstrap.cc:654] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.159126 17824 log.cc:824] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:37.159034 16807 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8a14c2447e45c5b148ff762bb55c00 (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.160177 16807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8a14c2447e45c5b148ff762bb55c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.160215 16806 tablet_service.cc:1467] Processing CreateTablet for tablet 0d27d6c2cc194773a648909806f4416b (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.160609 16938 tablet_service.cc:1467] Processing CreateTablet for tablet 452b5685e6ca4793a6628d700599bdbb (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.160765 16805 tablet_service.cc:1467] Processing CreateTablet for tablet 452b5685e6ca4793a6628d700599bdbb (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.160917 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452b5685e6ca4793a6628d700599bdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.161377 17202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d27d6c2cc194773a648909806f4416b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.165699 16805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452b5685e6ca4793a6628d700599bdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.166774 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d27d6c2cc194773a648909806f4416b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.172938 17827 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:37.174682 17824 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:37.174686 16541 tablet_service.cc:1467] Processing CreateTablet for tablet 452b5685e6ca4793a6628d700599bdbb (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.174901 17824 ts_tablet_manager.cc:1367] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.000s
I20240501 08:00:37.175446 16541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452b5685e6ca4793a6628d700599bdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.175460 17827 tablet_bootstrap.cc:654] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.176412 17827 log.cc:824] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:37.178637 17827 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:37.178817 17828 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:37.178886 17827 ts_tablet_manager.cc:1367] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20240501 08:00:37.178864 17824 raft_consensus.cc:348] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.179160 17824 raft_consensus.cc:374] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.179232 17824 raft_consensus.cc:729] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.179821 17824 consensus_queue.cc:260] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.180531 16543 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8a14c2447e45c5b148ff762bb55c00 (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.180903 16543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8a14c2447e45c5b148ff762bb55c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.180899 17824 ts_tablet_manager.cc:1398] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240501 08:00:37.181350 17824 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:37.173821 16542 tablet_service.cc:1467] Processing CreateTablet for tablet 0d27d6c2cc194773a648909806f4416b (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.184226 16542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d27d6c2cc194773a648909806f4416b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.185554 17828 tablet_bootstrap.cc:654] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.186142 17826 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:37.187579 17824 tablet_bootstrap.cc:654] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.187501 17827 raft_consensus.cc:348] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.187809 17827 raft_consensus.cc:374] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.187876 17827 raft_consensus.cc:729] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.188777 17827 consensus_queue.cc:260] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.188673 16674 tablet_service.cc:1467] Processing CreateTablet for tablet 0d27d6c2cc194773a648909806f4416b (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.189261 17828 log.cc:824] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:37.189958 17827 ts_tablet_manager.cc:1398] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.011s user 0.002s sys 0.004s
I20240501 08:00:37.190307 16673 tablet_service.cc:1467] Processing CreateTablet for tablet 452b5685e6ca4793a6628d700599bdbb (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.190444 17827 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:37.190606 16674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d27d6c2cc194773a648909806f4416b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.191295 17826 tablet_bootstrap.cc:654] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.191869 17827 tablet_bootstrap.cc:654] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.192507 17826 log.cc:824] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:37.193238 16673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452b5685e6ca4793a6628d700599bdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.193382 17827 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:37.193490 17827 ts_tablet_manager.cc:1367] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 08:00:37.194133 17827 raft_consensus.cc:348] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.194272 17827 raft_consensus.cc:374] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.194307 17827 raft_consensus.cc:729] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.194459 17827 consensus_queue.cc:260] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.194891 17827 ts_tablet_manager.cc:1398] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240501 08:00:37.195087 17827 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:37.195197 16675 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8a14c2447e45c5b148ff762bb55c00 (DEFAULT_TABLE table=rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.195499 16675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8a14c2447e45c5b148ff762bb55c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.196218 17827 tablet_bootstrap.cc:654] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.197647 17827 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:37.197758 17827 ts_tablet_manager.cc:1367] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:37.198298 17827 raft_consensus.cc:348] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.198498 17827 raft_consensus.cc:374] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.198549 17827 raft_consensus.cc:729] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.198700 17827 consensus_queue.cc:260] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.199019 17827 ts_tablet_manager.cc:1398] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:37.200994 17839 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:37.202569 17829 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:37.203444 17824 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:37.203562 17824 ts_tablet_manager.cc:1367] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.001s
I20240501 08:00:37.203675 17840 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:37.204322 17824 raft_consensus.cc:348] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.204473 17828 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:37.204483 17824 raft_consensus.cc:374] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.204521 17824 raft_consensus.cc:729] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.204689 17828 ts_tablet_manager.cc:1367] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.000s
I20240501 08:00:37.204671 17824 consensus_queue.cc:260] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.204977 17824 ts_tablet_manager.cc:1398] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240501 08:00:37.205176 17824 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:37.205360 17826 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:37.205619 17826 ts_tablet_manager.cc:1367] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20240501 08:00:37.206195 17824 tablet_bootstrap.cc:654] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.206218 17840 tablet_bootstrap.cc:654] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.207190 17840 log.cc:824] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:37.208652 17824 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:37.208788 17824 ts_tablet_manager.cc:1367] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 08:00:37.208684 17828 raft_consensus.cc:348] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.208968 17828 raft_consensus.cc:374] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.209038 17828 raft_consensus.cc:729] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
W20240501 08:00:37.209399 17270 tablet.cc:2369] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 08:00:37.209335 17824 raft_consensus.cc:348] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.209484 17824 raft_consensus.cc:374] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.209514 17824 raft_consensus.cc:729] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.209568 17828 consensus_queue.cc:260] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.209643 17824 consensus_queue.cc:260] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.209909 17839 tablet_bootstrap.cc:654] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.209964 17824 ts_tablet_manager.cc:1398] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:37.210558 17828 ts_tablet_manager.cc:1398] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240501 08:00:37.210830 17839 log.cc:824] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:37.210965 17828 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:37.212064 17828 tablet_bootstrap.cc:654] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.213440 17829 tablet_bootstrap.cc:654] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.213485 17840 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:37.213703 17840 ts_tablet_manager.cc:1367] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20240501 08:00:37.213500 17826 raft_consensus.cc:348] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.214008 17826 raft_consensus.cc:374] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.214107 17826 raft_consensus.cc:729] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.214577 17829 log.cc:824] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:37.214778 17826 consensus_queue.cc:260] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.216150 17826 ts_tablet_manager.cc:1398] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20240501 08:00:37.216657 17826 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:37.217147 17829 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:37.217377 17829 ts_tablet_manager.cc:1367] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20240501 08:00:37.218431 17826 tablet_bootstrap.cc:654] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.219455 17839 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:37.219686 17839 ts_tablet_manager.cc:1367] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.000s
I20240501 08:00:37.219756 17826 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:37.219844 17826 ts_tablet_manager.cc:1367] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:37.220300 17840 raft_consensus.cc:348] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.220474 17826 raft_consensus.cc:348] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.220595 17840 raft_consensus.cc:374] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.220618 17826 raft_consensus.cc:374] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.220650 17826 raft_consensus.cc:729] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.220659 17840 raft_consensus.cc:729] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.220799 17826 consensus_queue.cc:260] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.221215 17826 ts_tablet_manager.cc:1398] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240501 08:00:37.221426 17826 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:37.221336 17840 consensus_queue.cc:260] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.222422 17840 ts_tablet_manager.cc:1398] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20240501 08:00:37.222515 17826 tablet_bootstrap.cc:654] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.223901 17840 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:37.224129 17826 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:37.224218 17826 ts_tablet_manager.cc:1367] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 08:00:37.224241 17829 raft_consensus.cc:348] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.224519 17829 raft_consensus.cc:374] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.224574 17829 raft_consensus.cc:729] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.224706 17826 raft_consensus.cc:348] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.224812 17826 raft_consensus.cc:374] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.224839 17826 raft_consensus.cc:729] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.224951 17826 consensus_queue.cc:260] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.225132 17840 tablet_bootstrap.cc:654] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.225131 17829 consensus_queue.cc:260] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.225359 17826 ts_tablet_manager.cc:1398] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:37.226392 17840 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:37.226513 17840 ts_tablet_manager.cc:1367] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 08:00:37.227195 17840 raft_consensus.cc:348] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.227356 17840 raft_consensus.cc:374] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.227385 17840 raft_consensus.cc:729] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.227393 17828 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:37.227492 17828 ts_tablet_manager.cc:1367] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
I20240501 08:00:37.227514 17840 consensus_queue.cc:260] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.227918 17840 ts_tablet_manager.cc:1398] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240501 08:00:37.228154 17840 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:37.228150 17828 raft_consensus.cc:348] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.228277 17828 raft_consensus.cc:374] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.228309 17828 raft_consensus.cc:729] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.228444 17828 consensus_queue.cc:260] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.228730 17828 ts_tablet_manager.cc:1398] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240501 08:00:37.228925 17828 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:37.229239 17840 tablet_bootstrap.cc:654] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.229970 17828 tablet_bootstrap.cc:654] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.230620 17840 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:37.230729 17840 ts_tablet_manager.cc:1367] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:37.230798 17829 ts_tablet_manager.cc:1398] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20240501 08:00:37.231236 17840 raft_consensus.cc:348] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.231390 17840 raft_consensus.cc:374] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.231431 17840 raft_consensus.cc:729] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.231561 17840 consensus_queue.cc:260] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.231751 17829 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:37.231894 17840 ts_tablet_manager.cc:1398] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:37.233081 17828 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:37.233187 17828 ts_tablet_manager.cc:1367] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240501 08:00:37.233716 17828 raft_consensus.cc:348] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.233860 17828 raft_consensus.cc:374] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.233897 17828 raft_consensus.cc:729] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.234048 17828 consensus_queue.cc:260] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.234426 17828 ts_tablet_manager.cc:1398] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:37.235049 17829 tablet_bootstrap.cc:654] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.235710 17839 raft_consensus.cc:348] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.236066 17839 raft_consensus.cc:374] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.236146 17839 raft_consensus.cc:729] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.236721 17839 consensus_queue.cc:260] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
W20240501 08:00:37.237819 17138 tablet.cc:2369] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 08:00:37.237917 17839 ts_tablet_manager.cc:1398] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20240501 08:00:37.237877 17829 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:37.238143 17829 ts_tablet_manager.cc:1367] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20240501 08:00:37.238236 17839 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:37.238906 17829 raft_consensus.cc:348] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.239179 17829 raft_consensus.cc:374] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.239259 17829 raft_consensus.cc:729] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.239440 17839 tablet_bootstrap.cc:654] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.239562 17829 consensus_queue.cc:260] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.240105 17829 ts_tablet_manager.cc:1398] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:37.240360 17829 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:37.240846 17839 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:37.240967 17839 ts_tablet_manager.cc:1367] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:37.241596 17829 tablet_bootstrap.cc:654] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.241619 17839 raft_consensus.cc:348] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.241763 17839 raft_consensus.cc:374] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.241806 17839 raft_consensus.cc:729] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.241954 17839 consensus_queue.cc:260] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.242278 17839 ts_tablet_manager.cc:1398] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:37.242478 17839 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:37.242871 17829 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:37.242985 17829 ts_tablet_manager.cc:1367] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:37.243499 17839 tablet_bootstrap.cc:654] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.243463 17829 raft_consensus.cc:348] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.243620 17829 raft_consensus.cc:374] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.243664 17829 raft_consensus.cc:729] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.243803 17829 consensus_queue.cc:260] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.244161 17829 ts_tablet_manager.cc:1398] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:37.245046 17839 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:37.245148 17839 ts_tablet_manager.cc:1367] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:37.245649 17839 raft_consensus.cc:348] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.245800 17839 raft_consensus.cc:374] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.245834 17839 raft_consensus.cc:729] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.246030 17839 consensus_queue.cc:260] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.246361 17839 ts_tablet_manager.cc:1398] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:37.275023 17837 raft_consensus.cc:482] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:37.275189 17843 raft_consensus.cc:482] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:37.275274 17837 raft_consensus.cc:504] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.275390 17843 raft_consensus.cc:504] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.277135 17843 leader_election.cc:288] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:37.277148 17837 leader_election.cc:288] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:37.286317 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d27d6c2cc194773a648909806f4416b" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:37.286329 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d27d6c2cc194773a648909806f4416b" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:37.286777 16695 raft_consensus.cc:2454] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 0.
I20240501 08:00:37.286779 16959 raft_consensus.cc:2454] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 0.
W20240501 08:00:37.287111 17006 tablet.cc:2369] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 08:00:37.287117 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452b5685e6ca4793a6628d700599bdbb" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:37.287343 17223 raft_consensus.cc:2454] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 0.
I20240501 08:00:37.289964 16827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d27d6c2cc194773a648909806f4416b" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:37.290343 16827 raft_consensus.cc:2454] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 0.
I20240501 08:00:37.290583 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452b5685e6ca4793a6628d700599bdbb" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:37.290912 17355 raft_consensus.cc:2454] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 0.
I20240501 08:00:37.292106 17156 leader_election.cc:302] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, d5a55f6821d84c968a42f4a5c9ad07a7; no voters:
I20240501 08:00:37.292495 17837 raft_consensus.cc:2781] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:37.292577 17837 raft_consensus.cc:482] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:37.292618 17837 raft_consensus.cc:3037] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.292836 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452b5685e6ca4793a6628d700599bdbb" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:37.292994 16959 raft_consensus.cc:2454] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 0.
I20240501 08:00:37.293371 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452b5685e6ca4793a6628d700599bdbb" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:37.293529 16695 raft_consensus.cc:2454] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 0.
I20240501 08:00:37.293951 17837 raft_consensus.cc:504] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.293995 17025 leader_election.cc:302] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:37.294364 17843 raft_consensus.cc:2781] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:37.294451 17843 raft_consensus.cc:482] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:37.294498 17843 raft_consensus.cc:3037] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.294668 17837 leader_election.cc:288] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 1 election: Requested vote from peers 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:37.295753 17843 raft_consensus.cc:504] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.295971 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d27d6c2cc194773a648909806f4416b" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:37.295971 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d27d6c2cc194773a648909806f4416b" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:37.296109 16695 raft_consensus.cc:3037] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.296109 16959 raft_consensus.cc:3037] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.296469 17843 leader_election.cc:288] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 election: Requested vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:37.297029 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452b5685e6ca4793a6628d700599bdbb" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:37.297196 16826 raft_consensus.cc:3037] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.297289 16695 raft_consensus.cc:2454] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 1.
I20240501 08:00:37.297289 16959 raft_consensus.cc:2454] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 1.
I20240501 08:00:37.297976 16827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452b5685e6ca4793a6628d700599bdbb" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:37.298491 16826 raft_consensus.cc:2454] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 1.
I20240501 08:00:37.298956 16562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d27d6c2cc194773a648909806f4416b" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:37.299064 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d27d6c2cc194773a648909806f4416b" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:37.299211 16562 raft_consensus.cc:3037] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.299672 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452b5685e6ca4793a6628d700599bdbb" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:37.299862 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d27d6c2cc194773a648909806f4416b" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:37.300009 16826 raft_consensus.cc:3037] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.300024 16695 raft_consensus.cc:3037] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.300824 16562 raft_consensus.cc:2454] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 1.
I20240501 08:00:37.300899 16826 raft_consensus.cc:2454] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 1.
I20240501 08:00:37.301126 16695 raft_consensus.cc:2454] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 1.
I20240501 08:00:37.301892 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452b5685e6ca4793a6628d700599bdbb" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:37.302042 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d27d6c2cc194773a648909806f4416b" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:37.302091 17354 raft_consensus.cc:3037] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.302264 17091 raft_consensus.cc:2454] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 0.
I20240501 08:00:37.302294 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d27d6c2cc194773a648909806f4416b" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:37.302400 17355 raft_consensus.cc:2454] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 0.
I20240501 08:00:37.302654 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d27d6c2cc194773a648909806f4416b" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:37.302754 17353 raft_consensus.cc:3037] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.303081 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d27d6c2cc194773a648909806f4416b" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:37.303201 17090 raft_consensus.cc:3037] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.303180 17156 leader_election.cc:302] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, d5a55f6821d84c968a42f4a5c9ad07a7; no voters:
I20240501 08:00:37.303362 17354 raft_consensus.cc:2454] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 1.
I20240501 08:00:37.303614 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452b5685e6ca4793a6628d700599bdbb" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:37.303706 17353 raft_consensus.cc:2454] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 1.
I20240501 08:00:37.303730 17223 raft_consensus.cc:3037] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.303977 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452b5685e6ca4793a6628d700599bdbb" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:37.304105 16563 raft_consensus.cc:3037] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.304217 17026 leader_election.cc:302] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:37.304281 17090 raft_consensus.cc:2454] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 1.
I20240501 08:00:37.304627 17843 raft_consensus.cc:2781] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:37.304853 17843 raft_consensus.cc:686] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Becoming Leader. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Running, Role: LEADER
I20240501 08:00:37.305135 16562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452b5685e6ca4793a6628d700599bdbb" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:37.305233 17837 raft_consensus.cc:2781] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:37.305330 17837 raft_consensus.cc:686] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Becoming Leader. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Running, Role: LEADER
I20240501 08:00:37.305300 17843 consensus_queue.cc:237] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.305565 17223 raft_consensus.cc:2454] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 1.
I20240501 08:00:37.305627 17837 consensus_queue.cc:237] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.305121 16563 raft_consensus.cc:2454] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 1.
I20240501 08:00:37.306210 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "452b5685e6ca4793a6628d700599bdbb" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:37.306347 16959 raft_consensus.cc:3037] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.307677 16959 raft_consensus.cc:2454] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 1.
I20240501 08:00:37.309619 16419 catalog_manager.cc:5484] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: term changed from 0 to 1, leader changed from <none> to e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:37.309619 16420 catalog_manager.cc:5484] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } }
W20240501 08:00:37.342067 16874 tablet.cc:2369] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 08:00:37.356948 17855 raft_consensus.cc:482] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:37.357117 17855 raft_consensus.cc:504] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.358546 17855 leader_election.cc:288] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:37.364606 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:37.364606 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:37.364841 17090 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.364841 17353 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.365144 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:37.365314 16826 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.365626 16496 leader_election.cc:302] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, af70005d0c6c48bc9919cae6346f221b, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:37.365972 17855 raft_consensus.cc:2781] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:37.366045 17855 raft_consensus.cc:482] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:37.366079 17855 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.367139 17855 raft_consensus.cc:504] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.367566 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:37.367744 16959 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.368222 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:37.368294 17855 leader_election.cc:288] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Requested vote from peers af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:37.368397 16695 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.368885 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:37.369038 17090 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.369073 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:37.369098 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:37.369200 16695 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.369205 16959 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.369236 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:37.369361 16826 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.370154 17090 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.370154 16695 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.370261 16959 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.370296 16826 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.371011 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:37.371220 17353 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.371224 16499 leader_election.cc:302] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:37.371515 17855 raft_consensus.cc:2781] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:37.371613 17855 raft_consensus.cc:686] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Becoming Leader. State: Replica: eca6f08d57d1453784f726199474a671, State: Running, Role: LEADER
W20240501 08:00:37.371734 16742 tablet.cc:2369] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 08:00:37.372038 17855 consensus_queue.cc:237] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.372385 17353 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.372363 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:37.372499 17223 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.372499 17222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:37.372676 17222 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.374157 17222 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.376519 16420 catalog_manager.cc:5484] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 reported cstate change: term changed from 0 to 1, leader changed from <none> to eca6f08d57d1453784f726199474a671 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } }
W20240501 08:00:37.410806 17402 tablet.cc:2369] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 08:00:37.413398 16610 tablet.cc:2369] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 08:00:37.554844 17891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:37.555142 17891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:37.558518 17891 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240501 08:00:37.678129 16959 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.678128 17090 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.678903 17833 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.680547 16826 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.680541 16695 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.680624 17353 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.681365 16563 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.682440 16695 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.682400 17837 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.683019 17837 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.683197 17855 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.683852 17090 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.684011 16958 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.684300 16826 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.685173 17855 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.685302 17927 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.690498 17926 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.691538 17222 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.692709 17929 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.693171 17928 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.698851 17353 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.700642 17833 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.704020 17930 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.704668 17930 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.710667 17922 mvcc.cc:205] Tried to move back new op lower bound from 7022798592777977856 to 7022798591208652800. Current Snapshot: MvccSnapshot[applied={T|T < 7022798592719319040}]
I20240501 08:00:37.713770 16825 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.714771 17355 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.715219 16957 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.716235 16563 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.716482 17843 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.716985 17843 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.717566 17843 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.718909 17222 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.719828 16695 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:37.726513 17880 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.726958 17880 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.727263 17880 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:37.741608 17923 mvcc.cc:205] Tried to move back new op lower bound from 7022798592777977856 to 7022798591208652800. Current Snapshot: MvccSnapshot[applied={T|T < 7022798592719319040}]
I20240501 08:00:37.751271 17937 mvcc.cc:205] Tried to move back new op lower bound from 7022798592777977856 to 7022798591208652800. Current Snapshot: MvccSnapshot[applied={T|T < 7022798592719319040}]
I20240501 08:00:37.753129 17932 mvcc.cc:205] Tried to move back new op lower bound from 7022798592719319040 to 7022798591208652800. Current Snapshot: MvccSnapshot[applied={T|T < 7022798592719319040}]
I20240501 08:00:37.773644 17947 mvcc.cc:205] Tried to move back new op lower bound from 7022798592719319040 to 7022798591208652800. Current Snapshot: MvccSnapshot[applied={T|T < 7022798592719319040}]
I20240501 08:00:37.783679 17964 mvcc.cc:205] Tried to move back new op lower bound from 7022798592777977856 to 7022798591208652800. Current Snapshot: MvccSnapshot[applied={T|T < 7022798592777977856}]
I20240501 08:00:37.820839 17949 mvcc.cc:205] Tried to move back new op lower bound from 7022798592719319040 to 7022798591208652800. Current Snapshot: MvccSnapshot[applied={T|T < 7022798592719319040}]
INSERT report
rows total: 2000
time total: 269.1 ms
time per row: 0.13455 ms
I20240501 08:00:37.885228 16419 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59002:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 08:00:37.885844 16419 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_1 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 08:00:37.894018 17071 tablet_service.cc:1467] Processing CreateTablet for tablet b082e9dc1a0144978b1d47337be97b37 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.894403 17071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b082e9dc1a0144978b1d47337be97b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.894327 16937 tablet_service.cc:1467] Processing CreateTablet for tablet b082e9dc1a0144978b1d47337be97b37 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.894666 16937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b082e9dc1a0144978b1d47337be97b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.895022 17069 tablet_service.cc:1467] Processing CreateTablet for tablet 5e0b477de25646eda4f9f0e4921ec118 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.895290 17069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0b477de25646eda4f9f0e4921ec118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.896808 17999 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:37.898145 17999 tablet_bootstrap.cc:654] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.898247 17334 tablet_service.cc:1467] Processing CreateTablet for tablet b082e9dc1a0144978b1d47337be97b37 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.898619 17334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b082e9dc1a0144978b1d47337be97b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.898836 18000 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:37.899833 16542 tablet_service.cc:1467] Processing CreateTablet for tablet 5e0b477de25646eda4f9f0e4921ec118 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.899901 16674 tablet_service.cc:1467] Processing CreateTablet for tablet b082e9dc1a0144978b1d47337be97b37 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.900238 16542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0b477de25646eda4f9f0e4921ec118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.900225 18000 tablet_bootstrap.cc:654] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.900496 16674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b082e9dc1a0144978b1d47337be97b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.901455 16543 tablet_service.cc:1467] Processing CreateTablet for tablet b082e9dc1a0144978b1d47337be97b37 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.901721 16543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b082e9dc1a0144978b1d47337be97b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.902243 18000 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:37.902354 18000 ts_tablet_manager.cc:1367] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 08:00:37.902920 18000 raft_consensus.cc:348] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.903059 18000 raft_consensus.cc:374] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.903091 18000 raft_consensus.cc:729] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.903071 16806 tablet_service.cc:1467] Processing CreateTablet for tablet b082e9dc1a0144978b1d47337be97b37 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.903239 18000 consensus_queue.cc:260] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.903400 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b082e9dc1a0144978b1d47337be97b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.903532 18000 ts_tablet_manager.cc:1398] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:37.903743 18000 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:37.903981 16675 tablet_service.cc:1467] Processing CreateTablet for tablet 5e0b477de25646eda4f9f0e4921ec118 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.904299 16675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0b477de25646eda4f9f0e4921ec118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.904734 18000 tablet_bootstrap.cc:654] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.905658 18002 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:37.906667 18006 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:37.906981 18002 tablet_bootstrap.cc:654] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.907626 18006 tablet_bootstrap.cc:654] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.907645 17203 tablet_service.cc:1467] Processing CreateTablet for tablet b082e9dc1a0144978b1d47337be97b37 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.908002 17203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b082e9dc1a0144978b1d47337be97b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.908819 17999 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:37.908818 18006 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:37.908929 18006 ts_tablet_manager.cc:1367] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:37.908949 17999 ts_tablet_manager.cc:1367] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240501 08:00:37.909104 18000 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:37.909219 18000 ts_tablet_manager.cc:1367] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240501 08:00:37.909453 18006 raft_consensus.cc:348] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.909608 18006 raft_consensus.cc:374] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.909518 17999 raft_consensus.cc:348] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.909649 18006 raft_consensus.cc:729] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.909704 17999 raft_consensus.cc:374] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.909754 17999 raft_consensus.cc:729] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.909793 18006 consensus_queue.cc:260] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.909833 18000 raft_consensus.cc:348] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.909981 18000 raft_consensus.cc:374] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.910032 18000 raft_consensus.cc:729] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.909942 17999 consensus_queue.cc:260] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.910115 18006 ts_tablet_manager.cc:1398] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:37.910171 18000 consensus_queue.cc:260] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.910496 18000 ts_tablet_manager.cc:1398] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240501 08:00:37.910758 17334 tablet_service.cc:1467] Processing CreateTablet for tablet 5e0b477de25646eda4f9f0e4921ec118 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.910806 16937 tablet_service.cc:1467] Processing CreateTablet for tablet 5e0b477de25646eda4f9f0e4921ec118 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.911085 17334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0b477de25646eda4f9f0e4921ec118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.911110 16937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0b477de25646eda4f9f0e4921ec118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.913504 18004 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:37.914641 17201 tablet_service.cc:1467] Processing CreateTablet for tablet 5e0b477de25646eda4f9f0e4921ec118 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.914984 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0b477de25646eda4f9f0e4921ec118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.915331 18004 tablet_bootstrap.cc:654] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.915576 17999 ts_tablet_manager.cc:1398] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240501 08:00:37.915876 17999 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:37.916963 17999 tablet_bootstrap.cc:654] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.917135 18004 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:37.917271 18004 ts_tablet_manager.cc:1367] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240501 08:00:37.917824 18004 raft_consensus.cc:348] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.917987 18004 raft_consensus.cc:374] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.918037 18004 raft_consensus.cc:729] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.918202 18004 consensus_queue.cc:260] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.918562 18004 ts_tablet_manager.cc:1398] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240501 08:00:37.918670 18002 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:37.918769 18002 ts_tablet_manager.cc:1367] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240501 08:00:37.918792 18004 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:37.919270 18002 raft_consensus.cc:348] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.919395 18002 raft_consensus.cc:374] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.919428 18002 raft_consensus.cc:729] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.919556 18002 consensus_queue.cc:260] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.919852 18002 ts_tablet_manager.cc:1398] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:37.920065 18002 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:37.920017 18004 tablet_bootstrap.cc:654] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.921067 18002 tablet_bootstrap.cc:654] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.921178 16543 tablet_service.cc:1467] Processing CreateTablet for tablet addb23c607754d0aabee4c934f390251 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.921442 16543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addb23c607754d0aabee4c934f390251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.921687 16806 tablet_service.cc:1467] Processing CreateTablet for tablet 5e0b477de25646eda4f9f0e4921ec118 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.921880 18004 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:37.921983 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0b477de25646eda4f9f0e4921ec118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.921981 18004 ts_tablet_manager.cc:1367] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 08:00:37.922538 18004 raft_consensus.cc:348] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.922683 18004 raft_consensus.cc:374] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.922713 18004 raft_consensus.cc:729] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.922842 18004 consensus_queue.cc:260] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.923156 18004 ts_tablet_manager.cc:1398] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:00:37.924072 18002 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:37.924177 18002 ts_tablet_manager.cc:1367] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 08:00:37.924624 16807 tablet_service.cc:1467] Processing CreateTablet for tablet addb23c607754d0aabee4c934f390251 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.924669 18002 raft_consensus.cc:348] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.924779 18002 raft_consensus.cc:374] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.924815 18002 raft_consensus.cc:729] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.924922 16807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addb23c607754d0aabee4c934f390251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.924976 18002 consensus_queue.cc:260] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.925387 18002 ts_tablet_manager.cc:1398] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:37.925568 18002 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:37.926409 16937 tablet_service.cc:1467] Processing CreateTablet for tablet addb23c607754d0aabee4c934f390251 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.926554 18002 tablet_bootstrap.cc:654] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.926735 16937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addb23c607754d0aabee4c934f390251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.927500 16674 tablet_service.cc:1467] Processing CreateTablet for tablet addb23c607754d0aabee4c934f390251 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.927832 16674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addb23c607754d0aabee4c934f390251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.928030 18002 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:37.928155 18002 ts_tablet_manager.cc:1367] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:37.928663 18002 raft_consensus.cc:348] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.928794 18002 raft_consensus.cc:374] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.928822 18002 raft_consensus.cc:729] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.928951 18002 consensus_queue.cc:260] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.929255 18002 ts_tablet_manager.cc:1398] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:37.930111 18003 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:37.930347 17334 tablet_service.cc:1467] Processing CreateTablet for tablet addb23c607754d0aabee4c934f390251 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.930648 17334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addb23c607754d0aabee4c934f390251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.930640 18006 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:37.931193 18003 tablet_bootstrap.cc:654] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.931383 17069 tablet_service.cc:1467] Processing CreateTablet for tablet addb23c607754d0aabee4c934f390251 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.931604 18006 tablet_bootstrap.cc:654] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.931663 17069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addb23c607754d0aabee4c934f390251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.933005 17201 tablet_service.cc:1467] Processing CreateTablet for tablet addb23c607754d0aabee4c934f390251 (DEFAULT_TABLE table=rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:37.933337 18004 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:37.933705 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addb23c607754d0aabee4c934f390251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:37.934032 18000 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:37.934262 18003 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:37.934382 18003 ts_tablet_manager.cc:1367] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 08:00:37.934703 18004 tablet_bootstrap.cc:654] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.934904 18003 raft_consensus.cc:348] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.935045 18003 raft_consensus.cc:374] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.935063 18000 tablet_bootstrap.cc:654] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.935078 18003 raft_consensus.cc:729] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.935227 18003 consensus_queue.cc:260] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.935534 18003 ts_tablet_manager.cc:1398] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:37.935747 18003 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:37.936653 18012 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:37.936762 18003 tablet_bootstrap.cc:654] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.937398 18012 tablet_bootstrap.cc:654] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.938071 17961 raft_consensus.cc:482] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:37.938187 17961 raft_consensus.cc:504] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.938505 18004 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:37.938597 17999 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:37.938622 18004 ts_tablet_manager.cc:1367] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20240501 08:00:37.938699 17999 ts_tablet_manager.cc:1367] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240501 08:00:37.938959 17961 leader_election.cc:288] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:37.939128 18004 raft_consensus.cc:348] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.939266 18004 raft_consensus.cc:374] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.939296 18004 raft_consensus.cc:729] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.939211 17999 raft_consensus.cc:348] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.939342 17999 raft_consensus.cc:374] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.939376 17999 raft_consensus.cc:729] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.939431 18004 consensus_queue.cc:260] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.939453 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0b477de25646eda4f9f0e4921ec118" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:37.939518 17999 consensus_queue.cc:260] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.939757 18004 ts_tablet_manager.cc:1398] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:00:37.939822 17999 ts_tablet_manager.cc:1398] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:37.940043 17999 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:37.940835 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0b477de25646eda4f9f0e4921ec118" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:37.940997 16957 raft_consensus.cc:2454] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 0.
I20240501 08:00:37.941256 17222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0b477de25646eda4f9f0e4921ec118" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:37.941345 17999 tablet_bootstrap.cc:654] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.941448 18012 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:37.941620 18012 ts_tablet_manager.cc:1367] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 08:00:37.941752 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0b477de25646eda4f9f0e4921ec118" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:37.941906 17353 raft_consensus.cc:2454] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 0.
I20240501 08:00:37.942329 18012 raft_consensus.cc:348] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.942679 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0b477de25646eda4f9f0e4921ec118" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:37.942844 16563 raft_consensus.cc:2454] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 0.
I20240501 08:00:37.942842 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0b477de25646eda4f9f0e4921ec118" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:37.942535 18012 raft_consensus.cc:374] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.943159 18012 raft_consensus.cc:729] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.943393 18012 consensus_queue.cc:260] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
W20240501 08:00:37.943626 17025 leader_election.cc:341] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:37.943907 18012 ts_tablet_manager.cc:1398] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20240501 08:00:37.944286 17026 leader_election.cc:341] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:37.944440 18000 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:37.944540 18000 ts_tablet_manager.cc:1367] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20240501 08:00:37.944572 17024 leader_election.cc:302] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters: 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7
I20240501 08:00:37.944727 18003 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
W20240501 08:00:37.944753 17024 leader_election.cc:341] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:37.944829 18003 ts_tablet_manager.cc:1367] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240501 08:00:37.944847 18006 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:37.944952 18006 ts_tablet_manager.cc:1367] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240501 08:00:37.945009 17961 raft_consensus.cc:2781] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:37.945060 17961 raft_consensus.cc:482] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:37.945096 17961 raft_consensus.cc:3037] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.945236 18000 raft_consensus.cc:348] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.945362 18000 raft_consensus.cc:374] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.945396 18000 raft_consensus.cc:729] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.945339 18003 raft_consensus.cc:348] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.945391 18006 raft_consensus.cc:348] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.945477 18003 raft_consensus.cc:374] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.945502 18006 raft_consensus.cc:374] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.945506 18003 raft_consensus.cc:729] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.945534 18006 raft_consensus.cc:729] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.945534 18000 consensus_queue.cc:260] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.945641 18003 consensus_queue.cc:260] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.945856 18000 ts_tablet_manager.cc:1398] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 08:00:37.945946 18003 ts_tablet_manager.cc:1398] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:37.946166 18003 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:37.946198 17961 raft_consensus.cc:504] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.946395 18006 consensus_queue.cc:260] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.946866 17961 leader_election.cc:288] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 election: Requested vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:37.947006 18006 ts_tablet_manager.cc:1398] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:37.947260 18006 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:37.947391 18003 tablet_bootstrap.cc:654] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.947491 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0b477de25646eda4f9f0e4921ec118" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:37.947618 16695 raft_consensus.cc:3037] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.948515 18006 tablet_bootstrap.cc:654] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.948733 16695 raft_consensus.cc:2454] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 1.
I20240501 08:00:37.949056 17929 raft_consensus.cc:482] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:37.949260 17999 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:37.949175 17929 raft_consensus.cc:504] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.949376 17999 ts_tablet_manager.cc:1367] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240501 08:00:37.949916 17929 leader_election.cc:288] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:37.950003 17999 raft_consensus.cc:348] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.950328 17999 raft_consensus.cc:374] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.950376 17999 raft_consensus.cc:729] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.950608 17999 consensus_queue.cc:260] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.950838 18003 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:37.950996 18003 ts_tablet_manager.cc:1367] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240501 08:00:37.951110 17999 ts_tablet_manager.cc:1398] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:37.951610 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b082e9dc1a0144978b1d47337be97b37" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:37.951637 18003 raft_consensus.cc:348] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.951961 17353 raft_consensus.cc:2454] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.951990 18003 raft_consensus.cc:374] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.952121 18003 raft_consensus.cc:729] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.952335 18003 consensus_queue.cc:260] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.952414 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b082e9dc1a0144978b1d47337be97b37" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:37.952554 17090 raft_consensus.cc:2454] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.952674 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b082e9dc1a0144978b1d47337be97b37" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:37.952833 16957 raft_consensus.cc:2454] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.953009 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b082e9dc1a0144978b1d47337be97b37" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:37.953088 18003 ts_tablet_manager.cc:1398] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:37.953163 16825 raft_consensus.cc:2454] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.953169 16498 leader_election.cc:302] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:37.954224 18006 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:37.954339 18006 ts_tablet_manager.cc:1367] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240501 08:00:37.954308 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0b477de25646eda4f9f0e4921ec118" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:37.954434 16825 raft_consensus.cc:3037] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.954921 18006 raft_consensus.cc:348] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.955045 18006 raft_consensus.cc:374] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.955078 18006 raft_consensus.cc:729] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.955209 18006 consensus_queue.cc:260] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.955484 18006 ts_tablet_manager.cc:1398] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:37.955607 16825 raft_consensus.cc:2454] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 1.
I20240501 08:00:37.955708 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0b477de25646eda4f9f0e4921ec118" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:37.955708 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0b477de25646eda4f9f0e4921ec118" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:37.955837 16563 raft_consensus.cc:3037] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.955837 16957 raft_consensus.cc:3037] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.956413 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0b477de25646eda4f9f0e4921ec118" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:37.956521 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b082e9dc1a0144978b1d47337be97b37" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:37.956584 17353 raft_consensus.cc:3037] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.956648 16695 raft_consensus.cc:2454] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.956868 17222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b082e9dc1a0144978b1d47337be97b37" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:37.957154 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e0b477de25646eda4f9f0e4921ec118" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:37.957252 17223 raft_consensus.cc:3037] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.957235 16563 raft_consensus.cc:2454] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 1.
I20240501 08:00:37.957604 17353 raft_consensus.cc:2454] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 1.
I20240501 08:00:37.957932 16957 raft_consensus.cc:2454] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 1.
I20240501 08:00:37.958351 17223 raft_consensus.cc:2454] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 1.
I20240501 08:00:37.958393 17026 leader_election.cc:302] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:37.958743 17961 raft_consensus.cc:2781] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:37.958927 17961 raft_consensus.cc:686] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Becoming Leader. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Running, Role: LEADER
W20240501 08:00:37.958889 16498 leader_election.cc:341] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:37.959125 18012 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:37.959105 17961 consensus_queue.cc:237] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.959240 17929 raft_consensus.cc:2781] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:37.959295 17929 raft_consensus.cc:482] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:37.959342 17929 raft_consensus.cc:3037] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.960464 17929 raft_consensus.cc:504] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.960852 18012 tablet_bootstrap.cc:654] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.961170 17929 leader_election.cc:288] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Requested vote from peers e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:37.961830 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b082e9dc1a0144978b1d47337be97b37" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:37.961840 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b082e9dc1a0144978b1d47337be97b37" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:37.961977 17090 raft_consensus.cc:3037] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.961984 16695 raft_consensus.cc:3037] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.961943 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b082e9dc1a0144978b1d47337be97b37" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:37.962596 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b082e9dc1a0144978b1d47337be97b37" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:37.962736 16825 raft_consensus.cc:3037] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.963356 17090 raft_consensus.cc:2454] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.963356 16695 raft_consensus.cc:2454] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.963871 16825 raft_consensus.cc:2454] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
W20240501 08:00:37.964412 16498 leader_election.cc:341] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:37.964582 16421 catalog_manager.cc:5484] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: term changed from 0 to 1, leader changed from <none> to e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:37.964884 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b082e9dc1a0144978b1d47337be97b37" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:37.964974 16497 leader_election.cc:302] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters: 3d06b1f1890443da898fde2eb82daad3
I20240501 08:00:37.965020 16957 raft_consensus.cc:3037] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.965690 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b082e9dc1a0144978b1d47337be97b37" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:37.965829 17353 raft_consensus.cc:3037] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.966838 17353 raft_consensus.cc:2454] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.966980 16957 raft_consensus.cc:2454] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.967073 17929 raft_consensus.cc:2781] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:37.967175 17929 raft_consensus.cc:686] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Becoming Leader. State: Replica: eca6f08d57d1453784f726199474a671, State: Running, Role: LEADER
I20240501 08:00:37.967343 17929 consensus_queue.cc:237] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.967825 18012 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:37.967962 18012 ts_tablet_manager.cc:1367] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240501 08:00:37.968477 18012 raft_consensus.cc:348] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.968595 18012 raft_consensus.cc:374] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.968623 18012 raft_consensus.cc:729] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.968992 18012 consensus_queue.cc:260] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:37.969369 18012 ts_tablet_manager.cc:1398] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:37.969903 18012 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:37.970662 16419 catalog_manager.cc:5484] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 reported cstate change: term changed from 0 to 1, leader changed from <none> to eca6f08d57d1453784f726199474a671 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:37.970849 18012 tablet_bootstrap.cc:654] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:37.972994 18012 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:37.973117 18012 ts_tablet_manager.cc:1367] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 08:00:37.973678 18012 raft_consensus.cc:348] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.973831 18012 raft_consensus.cc:374] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:37.973865 18012 raft_consensus.cc:729] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:37.973994 18012 consensus_queue.cc:260] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.974310 18012 ts_tablet_manager.cc:1398] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:37.983268 17929 raft_consensus.cc:482] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:37.983424 17929 raft_consensus.cc:504] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.984113 17929 leader_election.cc:288] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:37.984447 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:37.984601 16695 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.984622 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:37.984723 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:37.984802 16825 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.984778 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:37.984844 17223 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.984908 17353 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.985067 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:37.985213 16957 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.985270 16498 leader_election.cc:302] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:37.985463 17929 raft_consensus.cc:2781] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:37.985631 17929 raft_consensus.cc:482] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:37.985764 17929 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.985929 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:37.986056 17090 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:37.986886 17929 raft_consensus.cc:504] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.987506 17929 leader_election.cc:288] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Requested vote from peers 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:37.988005 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:37.988149 16695 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.988308 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:37.988386 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:37.988476 16957 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.988627 16825 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.988709 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:37.988840 17090 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.989403 16695 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.989578 16825 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.989612 16957 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.990119 17090 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.990401 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:37.990463 16499 leader_election.cc:302] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:37.990571 17223 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.990566 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:37.990716 17353 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:37.990710 17929 raft_consensus.cc:2781] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:37.990831 17929 raft_consensus.cc:686] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Becoming Leader. State: Replica: eca6f08d57d1453784f726199474a671, State: Running, Role: LEADER
I20240501 08:00:37.991200 17929 consensus_queue.cc:237] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:37.991513 17223 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.991756 17353 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:37.993849 16421 catalog_manager.cc:5484] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 reported cstate change: term changed from 0 to 1, leader changed from <none> to eca6f08d57d1453784f726199474a671 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } }
W20240501 08:00:38.120761 18030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:38.121052 18030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:38.124547 18030 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240501 08:00:38.236682 17090 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.237038 17223 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.238674 16825 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:00:38.238772 16695 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:00:38.239159 17930 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.239991 17931 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.241082 16957 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.241439 17353 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.241806 17961 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.246649 16695 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.247216 17223 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.247833 16825 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.248301 17956 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.248754 17956 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.253355 16958 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.253440 17354 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.254102 16959 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.254350 18061 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.254809 18061 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.254930 16563 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.256001 17353 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 08:00:38.256669 17090 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.256709 16825 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.257367 17930 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.257447 17880 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.257761 17930 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.257790 17880 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.258078 17930 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.258397 17930 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.258913 17843 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.259649 17850 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.260156 17222 raft_consensus.cc:3037] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.261415 17222 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.281363 16693 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.281466 17855 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.281675 17928 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.283550 17850 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 218.908 ms
time per row: 0.109454 ms
I20240501 08:00:38.401177 16415 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59002:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 08:00:38.401806 16415 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_2 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 08:00:38.412127 17335 tablet_service.cc:1467] Processing CreateTablet for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.412324 16939 tablet_service.cc:1467] Processing CreateTablet for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.412595 16939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.412596 17335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.412117 17334 tablet_service.cc:1467] Processing CreateTablet for tablet ab47ecf7f96c4f668750a2bb0bddb1ef (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.413352 17334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab47ecf7f96c4f668750a2bb0bddb1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.414844 18004 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:38.415035 16938 tablet_service.cc:1467] Processing CreateTablet for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.415302 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b060d9fcf294f55b5ccdd3edf5b2886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.415896 18004 tablet_bootstrap.cc:654] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.417805 17201 tablet_service.cc:1467] Processing CreateTablet for tablet ab47ecf7f96c4f668750a2bb0bddb1ef (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.418119 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab47ecf7f96c4f668750a2bb0bddb1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.420428 17202 tablet_service.cc:1467] Processing CreateTablet for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.420711 17202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b060d9fcf294f55b5ccdd3edf5b2886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.422461 18012 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:38.422883 17333 tablet_service.cc:1467] Processing CreateTablet for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.423117 17203 tablet_service.cc:1467] Processing CreateTablet for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.423214 17333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b060d9fcf294f55b5ccdd3edf5b2886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.423408 17203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.423497 18012 tablet_bootstrap.cc:654] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.424243 17999 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:38.425037 18012 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:38.425139 18012 ts_tablet_manager.cc:1367] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:38.425427 17999 tablet_bootstrap.cc:654] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.425619 18012 raft_consensus.cc:348] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.425770 18012 raft_consensus.cc:374] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.425796 18012 raft_consensus.cc:729] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.425941 18012 consensus_queue.cc:260] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.426239 18012 ts_tablet_manager.cc:1398] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:38.426281 16937 tablet_service.cc:1467] Processing CreateTablet for tablet ab47ecf7f96c4f668750a2bb0bddb1ef (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.426558 16937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab47ecf7f96c4f668750a2bb0bddb1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.426683 17999 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:38.426780 17999 ts_tablet_manager.cc:1367] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:38.427347 17999 raft_consensus.cc:348] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.427469 17999 raft_consensus.cc:374] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.427500 17999 raft_consensus.cc:729] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.427692 17999 consensus_queue.cc:260] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.428017 17999 ts_tablet_manager.cc:1398] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:38.428182 17999 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:38.429193 17999 tablet_bootstrap.cc:654] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.430045 16674 tablet_service.cc:1467] Processing CreateTablet for tablet ab47ecf7f96c4f668750a2bb0bddb1ef (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.430315 16674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab47ecf7f96c4f668750a2bb0bddb1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.431344 16807 tablet_service.cc:1467] Processing CreateTablet for tablet ab47ecf7f96c4f668750a2bb0bddb1ef (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.431696 16807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab47ecf7f96c4f668750a2bb0bddb1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.432206 17069 tablet_service.cc:1467] Processing CreateTablet for tablet ab47ecf7f96c4f668750a2bb0bddb1ef (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.432528 17069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab47ecf7f96c4f668750a2bb0bddb1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.433324 16675 tablet_service.cc:1467] Processing CreateTablet for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.433864 16675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.434724 16806 tablet_service.cc:1467] Processing CreateTablet for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.435030 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.435225 17071 tablet_service.cc:1467] Processing CreateTablet for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.435489 17071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.436287 16673 tablet_service.cc:1467] Processing CreateTablet for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.437741 16805 tablet_service.cc:1467] Processing CreateTablet for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.437918 18000 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:38.437952 17070 tablet_service.cc:1467] Processing CreateTablet for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.438051 16805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b060d9fcf294f55b5ccdd3edf5b2886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.438170 17070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b060d9fcf294f55b5ccdd3edf5b2886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.438864 18000 tablet_bootstrap.cc:654] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.440177 18006 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:38.441046 18006 tablet_bootstrap.cc:654] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.441219 18004 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:38.441336 18004 ts_tablet_manager.cc:1367] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.002s
I20240501 08:00:38.441828 18004 raft_consensus.cc:348] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.441977 18004 raft_consensus.cc:374] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.442005 18004 raft_consensus.cc:729] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.442148 18004 consensus_queue.cc:260] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.442251 18006 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:38.442353 18006 ts_tablet_manager.cc:1367] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:38.442445 18004 ts_tablet_manager.cc:1398] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:38.442646 18004 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:38.442821 18006 raft_consensus.cc:348] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.442951 18006 raft_consensus.cc:374] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.442981 18006 raft_consensus.cc:729] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.443109 18006 consensus_queue.cc:260] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.443396 18006 ts_tablet_manager.cc:1398] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:38.444394 16541 tablet_service.cc:1467] Processing CreateTablet for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.444602 18006 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:38.444717 16541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b060d9fcf294f55b5ccdd3edf5b2886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.445698 18006 tablet_bootstrap.cc:654] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.446967 18006 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:38.447078 18006 ts_tablet_manager.cc:1367] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:38.447345 16542 tablet_service.cc:1467] Processing CreateTablet for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.447635 16542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.447595 18006 raft_consensus.cc:348] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.447746 18006 raft_consensus.cc:374] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.447780 18006 raft_consensus.cc:729] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.447921 18006 consensus_queue.cc:260] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.448287 18006 ts_tablet_manager.cc:1398] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:38.448479 18006 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:38.450139 16543 tablet_service.cc:1467] Processing CreateTablet for tablet ab47ecf7f96c4f668750a2bb0bddb1ef (DEFAULT_TABLE table=rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:38.450470 16543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab47ecf7f96c4f668750a2bb0bddb1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.450565 18006 tablet_bootstrap.cc:654] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.452208 18094 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:38.453495 18012 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:38.453866 18003 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:38.454254 18004 tablet_bootstrap.cc:654] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.454572 18012 tablet_bootstrap.cc:654] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.454978 18003 tablet_bootstrap.cc:654] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.455835 16673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b060d9fcf294f55b5ccdd3edf5b2886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:38.456735 18000 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:38.456853 18000 ts_tablet_manager.cc:1367] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.001s
I20240501 08:00:38.456871 17999 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:38.456962 17999 ts_tablet_manager.cc:1367] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20240501 08:00:38.457335 18000 raft_consensus.cc:348] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.457464 18000 raft_consensus.cc:374] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.457492 18000 raft_consensus.cc:729] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.457439 17999 raft_consensus.cc:348] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.457562 17999 raft_consensus.cc:374] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.457590 17999 raft_consensus.cc:729] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.457631 18000 consensus_queue.cc:260] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.457741 17999 consensus_queue.cc:260] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.458043 17999 ts_tablet_manager.cc:1398] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:38.458225 17999 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:38.458963 18094 tablet_bootstrap.cc:654] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.459303 17999 tablet_bootstrap.cc:654] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.460089 18000 ts_tablet_manager.cc:1398] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 08:00:38.460315 18000 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:38.460597 18006 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:38.460716 18006 ts_tablet_manager.cc:1367] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240501 08:00:38.461177 18000 tablet_bootstrap.cc:654] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.461230 18006 raft_consensus.cc:348] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.461364 18006 raft_consensus.cc:374] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.461401 18006 raft_consensus.cc:729] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.461541 18006 consensus_queue.cc:260] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.461849 18006 ts_tablet_manager.cc:1398] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:38.464108 17847 raft_consensus.cc:482] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:38.464244 17847 raft_consensus.cc:504] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.464422 18003 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:38.464542 18003 ts_tablet_manager.cc:1367] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240501 08:00:38.464888 18004 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:38.464988 18004 ts_tablet_manager.cc:1367] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.002s
I20240501 08:00:38.465093 18003 raft_consensus.cc:348] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.465395 18003 raft_consensus.cc:374] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.465526 18003 raft_consensus.cc:729] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.465884 18003 consensus_queue.cc:260] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.465921 18004 raft_consensus.cc:348] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.466176 18004 raft_consensus.cc:374] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.466236 18004 raft_consensus.cc:729] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.466636 18003 ts_tablet_manager.cc:1398] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:38.466552 18004 consensus_queue.cc:260] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.467247 18004 ts_tablet_manager.cc:1398] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240501 08:00:38.467262 17847 leader_election.cc:288] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:38.467698 18004 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:38.469918 18004 tablet_bootstrap.cc:654] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.470052 18003 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:38.471298 18012 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:38.471414 18012 ts_tablet_manager.cc:1367] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240501 08:00:38.471441 18003 tablet_bootstrap.cc:654] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.471909 18012 raft_consensus.cc:348] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.472093 18012 raft_consensus.cc:374] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.472121 18012 raft_consensus.cc:729] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.472267 18012 consensus_queue.cc:260] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.472579 18012 ts_tablet_manager.cc:1398] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:00:38.472795 18012 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:38.473769 18012 tablet_bootstrap.cc:654] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.475553 17999 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:38.475689 17999 ts_tablet_manager.cc:1367] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240501 08:00:38.476231 17999 raft_consensus.cc:348] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.476367 17999 raft_consensus.cc:374] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.476397 17999 raft_consensus.cc:729] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.476560 18094 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:38.476541 17999 consensus_queue.cc:260] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.476686 18094 ts_tablet_manager.cc:1367] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240501 08:00:38.476852 17999 ts_tablet_manager.cc:1398] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:38.477239 18094 raft_consensus.cc:348] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.477394 18094 raft_consensus.cc:374] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.477428 18094 raft_consensus.cc:729] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.477658 18000 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:38.477768 18000 ts_tablet_manager.cc:1367] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240501 08:00:38.478242 18000 raft_consensus.cc:348] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.478379 18000 raft_consensus.cc:374] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.478410 18000 raft_consensus.cc:729] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.478574 18000 consensus_queue.cc:260] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.478603 18094 consensus_queue.cc:260] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.478883 18000 ts_tablet_manager.cc:1398] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:38.479064 18000 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:38.479075 18094 ts_tablet_manager.cc:1398] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:38.479259 18094 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:38.480118 18000 tablet_bootstrap.cc:654] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.480253 18094 tablet_bootstrap.cc:654] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.481658 18000 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:38.481792 18000 ts_tablet_manager.cc:1367] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 08:00:38.482250 18000 raft_consensus.cc:348] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.482374 18000 raft_consensus.cc:374] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.482399 18000 raft_consensus.cc:729] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.482524 18000 consensus_queue.cc:260] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.482805 18000 ts_tablet_manager.cc:1398] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:00:38.485970 18012 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:38.486146 18012 ts_tablet_manager.cc:1367] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20240501 08:00:38.486719 18004 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:38.486831 18004 ts_tablet_manager.cc:1367] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20240501 08:00:38.486743 18012 raft_consensus.cc:348] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.486925 18012 raft_consensus.cc:374] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.486960 18012 raft_consensus.cc:729] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.487131 18012 consensus_queue.cc:260] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.487300 18004 raft_consensus.cc:348] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.487443 18004 raft_consensus.cc:374] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.487437 18012 ts_tablet_manager.cc:1398] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 08:00:38.487471 18004 raft_consensus.cc:729] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.487600 18004 consensus_queue.cc:260] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.487886 18004 ts_tablet_manager.cc:1398] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:38.491709 18003 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:38.491859 18003 ts_tablet_manager.cc:1367] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20240501 08:00:38.492595 18003 raft_consensus.cc:348] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.492946 18003 raft_consensus.cc:374] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.493002 18003 raft_consensus.cc:729] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.493266 18003 consensus_queue.cc:260] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.493381 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:38.493636 16825 raft_consensus.cc:2454] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 0.
I20240501 08:00:38.494546 18003 ts_tablet_manager.cc:1398] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:38.494797 18003 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:38.495958 18003 tablet_bootstrap.cc:654] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.496855 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:38.497046 16694 raft_consensus.cc:2454] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 0.
I20240501 08:00:38.497730 18003 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:38.497857 18003 ts_tablet_manager.cc:1367] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:38.497964 18094 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:38.498103 18094 ts_tablet_manager.cc:1367] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240501 08:00:38.498405 18003 raft_consensus.cc:348] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.498584 18003 raft_consensus.cc:374] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.498631 18003 raft_consensus.cc:729] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.498632 18094 raft_consensus.cc:348] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.498780 18094 raft_consensus.cc:374] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.498817 18094 raft_consensus.cc:729] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.498806 18003 consensus_queue.cc:260] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.498975 18094 consensus_queue.cc:260] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.499266 18003 ts_tablet_manager.cc:1398] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:38.499370 18094 ts_tablet_manager.cc:1398] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:38.499576 18094 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:38.501019 18094 tablet_bootstrap.cc:654] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:38.503026 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:38.503211 16563 raft_consensus.cc:2454] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 0.
I20240501 08:00:38.503340 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:38.503489 17223 raft_consensus.cc:2454] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 0.
I20240501 08:00:38.503901 16894 leader_election.cc:302] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, d5a55f6821d84c968a42f4a5c9ad07a7; no voters:
I20240501 08:00:38.503968 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:38.504118 17090 raft_consensus.cc:2454] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 0.
I20240501 08:00:38.504941 17847 raft_consensus.cc:2781] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:38.504904 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:38.505005 17847 raft_consensus.cc:482] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:38.505040 17847 raft_consensus.cc:3037] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.505048 17355 raft_consensus.cc:2454] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 0.
I20240501 08:00:38.505810 18094 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:38.505906 18094 ts_tablet_manager.cc:1367] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20240501 08:00:38.506001 17847 raft_consensus.cc:504] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.506376 18094 raft_consensus.cc:348] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.506506 18094 raft_consensus.cc:374] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:38.506583 17847 leader_election.cc:288] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [CANDIDATE]: Term 1 election: Requested vote from peers eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:38.507174 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:38.507207 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:38.507227 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:38.507308 17355 raft_consensus.cc:3037] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.507334 17090 raft_consensus.cc:3037] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.507339 17223 raft_consensus.cc:3037] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.508136 18094 raft_consensus.cc:729] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:38.508402 17355 raft_consensus.cc:2454] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 1.
I20240501 08:00:38.508402 17090 raft_consensus.cc:2454] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 1.
I20240501 08:00:38.508420 17223 raft_consensus.cc:2454] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 1.
I20240501 08:00:38.508419 18094 consensus_queue.cc:260] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.508841 18094 ts_tablet_manager.cc:1398] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 08:00:38.509016 16894 leader_election.cc:302] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:38.509073 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:38.509192 16563 raft_consensus.cc:3037] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.509218 17847 raft_consensus.cc:2781] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:38.509430 17847 raft_consensus.cc:686] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Becoming Leader. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Running, Role: LEADER
I20240501 08:00:38.509807 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:38.509821 17847 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.509944 16694 raft_consensus.cc:3037] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.510243 16563 raft_consensus.cc:2454] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 1.
I20240501 08:00:38.511093 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:38.511253 16825 raft_consensus.cc:3037] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.512257 16825 raft_consensus.cc:2454] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 1.
I20240501 08:00:38.512656 16694 raft_consensus.cc:2454] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 1.
I20240501 08:00:38.513778 16416 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:38.540302 17859 raft_consensus.cc:482] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:38.540473 17859 raft_consensus.cc:504] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.541910 17859 leader_election.cc:288] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:38.542428 17853 raft_consensus.cc:482] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:38.542560 17853 raft_consensus.cc:504] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.544926 17853 leader_election.cc:288] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:38.547709 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:38.547988 17090 raft_consensus.cc:2454] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 0.
I20240501 08:00:38.548924 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:38.549088 16563 raft_consensus.cc:2454] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 0.
I20240501 08:00:38.550441 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:38.550629 16958 raft_consensus.cc:2454] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 0.
I20240501 08:00:38.551018 16630 leader_election.cc:302] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 64196487691e4623b690b2869aad57f7, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:38.551364 17859 raft_consensus.cc:2781] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:38.551424 17859 raft_consensus.cc:482] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:38.551378 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:38.551456 17859 raft_consensus.cc:3037] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.551513 16825 raft_consensus.cc:2454] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 0.
I20240501 08:00:38.552755 17859 raft_consensus.cc:504] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.553297 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:38.553452 17859 leader_election.cc:288] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [CANDIDATE]: Term 1 election: Requested vote from peers eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:38.553462 17355 raft_consensus.cc:2454] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 0.
I20240501 08:00:38.554234 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:38.554383 16825 raft_consensus.cc:3037] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.554431 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:38.554555 16563 raft_consensus.cc:3037] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.555454 16825 raft_consensus.cc:2454] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 1.
I20240501 08:00:38.555456 16563 raft_consensus.cc:2454] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 1.
I20240501 08:00:38.555764 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:38.555905 17090 raft_consensus.cc:3037] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.556533 18080 raft_consensus.cc:482] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:38.556658 18080 raft_consensus.cc:504] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.556999 17090 raft_consensus.cc:2454] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 1.
I20240501 08:00:38.557356 18080 leader_election.cc:288] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:38.558032 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:38.558058 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:38.558183 17355 raft_consensus.cc:3037] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.558184 16958 raft_consensus.cc:3037] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.558579 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:38.558944 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:38.559253 17355 raft_consensus.cc:2454] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 1.
I20240501 08:00:38.559247 16958 raft_consensus.cc:2454] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 1.
I20240501 08:00:38.559532 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:38.559729 16825 raft_consensus.cc:2379] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d06b1f1890443da898fde2eb82daad3 in current term 1: Already voted for candidate 64196487691e4623b690b2869aad57f7 in this term.
I20240501 08:00:38.560050 16631 leader_election.cc:302] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:38.560089 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:38.560254 16694 raft_consensus.cc:2379] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d06b1f1890443da898fde2eb82daad3 in current term 1: Already voted for candidate 64196487691e4623b690b2869aad57f7 in this term.
I20240501 08:00:38.560398 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:38.560592 16563 raft_consensus.cc:2379] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d06b1f1890443da898fde2eb82daad3 in current term 1: Already voted for candidate 64196487691e4623b690b2869aad57f7 in this term.
I20240501 08:00:38.560698 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:38.560850 17223 raft_consensus.cc:2454] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 0.
I20240501 08:00:38.560904 17859 raft_consensus.cc:2781] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:38.561051 17221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:38.561151 17221 raft_consensus.cc:3037] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.561169 17859 raft_consensus.cc:686] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Becoming Leader. State: Replica: 64196487691e4623b690b2869aad57f7, State: Running, Role: LEADER
I20240501 08:00:38.561555 17859 consensus_queue.cc:237] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.562297 17221 raft_consensus.cc:2454] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 1.
I20240501 08:00:38.562597 17156 leader_election.cc:302] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3; no voters: 1160c6e72055456992b5d8bf5f3b45c0, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7, eca6f08d57d1453784f726199474a671
I20240501 08:00:38.565128 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab47ecf7f96c4f668750a2bb0bddb1ef" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:38.565305 17355 raft_consensus.cc:2454] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:38.565505 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab47ecf7f96c4f668750a2bb0bddb1ef" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:38.565655 16563 raft_consensus.cc:2454] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:38.565536 16416 catalog_manager.cc:5484] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64196487691e4623b690b2869aad57f7 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:38.566146 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab47ecf7f96c4f668750a2bb0bddb1ef" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:38.566283 16694 raft_consensus.cc:2454] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:38.566331 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b060d9fcf294f55b5ccdd3edf5b2886" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:38.566509 17090 raft_consensus.cc:2379] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d06b1f1890443da898fde2eb82daad3 in current term 1: Already voted for candidate 64196487691e4623b690b2869aad57f7 in this term.
I20240501 08:00:38.566473 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab47ecf7f96c4f668750a2bb0bddb1ef" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:38.566607 16958 raft_consensus.cc:2454] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:38.566612 16761 leader_election.cc:302] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:38.566864 18080 raft_consensus.cc:2726] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:00:38.566902 17853 raft_consensus.cc:2781] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:38.566946 17853 raft_consensus.cc:482] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:38.566977 17853 raft_consensus.cc:3037] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.568029 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab47ecf7f96c4f668750a2bb0bddb1ef" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:38.568186 17090 raft_consensus.cc:2454] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:38.568161 17853 raft_consensus.cc:504] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.568833 17853 leader_election.cc:288] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 election: Requested vote from peers eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:38.569123 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab47ecf7f96c4f668750a2bb0bddb1ef" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:38.569262 16563 raft_consensus.cc:3037] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.569651 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab47ecf7f96c4f668750a2bb0bddb1ef" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:38.569784 17355 raft_consensus.cc:3037] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.569996 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab47ecf7f96c4f668750a2bb0bddb1ef" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:38.570155 16958 raft_consensus.cc:3037] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.570333 16563 raft_consensus.cc:2454] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:38.570775 17355 raft_consensus.cc:2454] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:38.571064 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab47ecf7f96c4f668750a2bb0bddb1ef" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:38.571185 16958 raft_consensus.cc:2454] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:38.571198 16694 raft_consensus.cc:3037] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.571265 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab47ecf7f96c4f668750a2bb0bddb1ef" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:38.571377 17090 raft_consensus.cc:3037] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.571758 16762 leader_election.cc:302] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:38.572031 17853 raft_consensus.cc:2781] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:38.572289 17853 raft_consensus.cc:686] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Becoming Leader. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Running, Role: LEADER
I20240501 08:00:38.572300 16694 raft_consensus.cc:2454] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:38.572258 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab47ecf7f96c4f668750a2bb0bddb1ef" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:38.572415 17223 raft_consensus.cc:3037] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:38.572695 17853 consensus_queue.cc:237] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:38.572937 17090 raft_consensus.cc:2454] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:38.572280 17221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab47ecf7f96c4f668750a2bb0bddb1ef" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:38.573448 17223 raft_consensus.cc:2454] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:38.576237 16416 catalog_manager.cc:5484] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } }
W20240501 08:00:38.736225 18141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:38.736524 18141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:38.739898 18141 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240501 08:00:38.889128 16958 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.889128 17223 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.889741 16694 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.889868 17090 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.890379 16563 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.890345 17853 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.891634 17355 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.891574 18139 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.891966 18139 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.892493 18139 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.892791 18139 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.893024 18139 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.893036 17355 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.893260 16958 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.893638 17859 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.893878 17223 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.894071 17859 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.897172 17091 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.898290 18137 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.898748 18137 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.902237 16562 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.902441 16825 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.903162 18137 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.903604 18137 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.910934 17089 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.911129 17353 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.912068 16561 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.912190 17847 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.914570 18119 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.915042 18119 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.919310 16695 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.920406 16827 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.921867 17847 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.922286 17847 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:38.936015 17221 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:38.943521 18119 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 275.489 ms
time per row: 0.137744 ms
I20240501 08:00:39.074426 16413 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59002:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 08:00:39.075064 16413 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_3 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 08:00:39.081630 16673 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae979c5adf44f3e8590ef06c71bf904 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.082015 16673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae979c5adf44f3e8590ef06c71bf904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.082926 16675 tablet_service.cc:1467] Processing CreateTablet for tablet bcce59ab1c5947efb63e2839e30ccc72 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.083205 16675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcce59ab1c5947efb63e2839e30ccc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.084555 17070 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae979c5adf44f3e8590ef06c71bf904 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.084673 18247 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:39.084885 17070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae979c5adf44f3e8590ef06c71bf904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.085006 16543 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae979c5adf44f3e8590ef06c71bf904 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.085268 16543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae979c5adf44f3e8590ef06c71bf904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.085613 18247 tablet_bootstrap.cc:654] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.086333 16675 tablet_service.cc:1467] Processing CreateTablet for tablet 7c25b2699528442f8e2993efbac96bbd (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.086529 16675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c25b2699528442f8e2993efbac96bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.087309 18249 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:39.087891 16937 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae979c5adf44f3e8590ef06c71bf904 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.088240 16937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae979c5adf44f3e8590ef06c71bf904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.088290 18249 tablet_bootstrap.cc:654] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.088563 16541 tablet_service.cc:1467] Processing CreateTablet for tablet 7c25b2699528442f8e2993efbac96bbd (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.088826 16541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c25b2699528442f8e2993efbac96bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.089030 18250 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:39.089926 18250 tablet_bootstrap.cc:654] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.090548 18253 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:39.091141 17071 tablet_service.cc:1467] Processing CreateTablet for tablet bcce59ab1c5947efb63e2839e30ccc72 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.091398 18253 tablet_bootstrap.cc:654] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.091435 17071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcce59ab1c5947efb63e2839e30ccc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.092604 18253 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:39.092741 18253 ts_tablet_manager.cc:1367] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:39.093155 18253 raft_consensus.cc:348] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.093276 18253 raft_consensus.cc:374] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.093298 18253 raft_consensus.cc:729] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.093425 18253 consensus_queue.cc:260] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.093725 18253 ts_tablet_manager.cc:1398] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:39.093840 17069 tablet_service.cc:1467] Processing CreateTablet for tablet 7c25b2699528442f8e2993efbac96bbd (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.094094 17069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c25b2699528442f8e2993efbac96bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.094306 18250 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:39.094411 18250 ts_tablet_manager.cc:1367] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 08:00:39.094938 18250 raft_consensus.cc:348] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.095091 18250 raft_consensus.cc:374] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.095127 18250 raft_consensus.cc:729] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.095260 18250 consensus_queue.cc:260] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.095722 18250 ts_tablet_manager.cc:1398] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:39.095917 18250 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:39.096364 16805 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae979c5adf44f3e8590ef06c71bf904 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.096645 16805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae979c5adf44f3e8590ef06c71bf904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.096890 18250 tablet_bootstrap.cc:654] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.098502 18250 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:39.098666 18250 ts_tablet_manager.cc:1367] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:39.099200 16807 tablet_service.cc:1467] Processing CreateTablet for tablet bcce59ab1c5947efb63e2839e30ccc72 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.099211 18250 raft_consensus.cc:348] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.099399 18250 raft_consensus.cc:374] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.099455 18250 raft_consensus.cc:729] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.099493 16807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcce59ab1c5947efb63e2839e30ccc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.099618 18250 consensus_queue.cc:260] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.100080 18250 ts_tablet_manager.cc:1398] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:39.101688 17202 tablet_service.cc:1467] Processing CreateTablet for tablet 7c25b2699528442f8e2993efbac96bbd (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.101994 17202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c25b2699528442f8e2993efbac96bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.103974 18257 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:39.104188 17203 tablet_service.cc:1467] Processing CreateTablet for tablet bcce59ab1c5947efb63e2839e30ccc72 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.104472 17203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcce59ab1c5947efb63e2839e30ccc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.104952 18257 tablet_bootstrap.cc:654] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.106052 18257 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:39.106160 18257 ts_tablet_manager.cc:1367] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:39.106374 16806 tablet_service.cc:1467] Processing CreateTablet for tablet 7c25b2699528442f8e2993efbac96bbd (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.106607 18257 raft_consensus.cc:348] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.106678 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c25b2699528442f8e2993efbac96bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.106758 18257 raft_consensus.cc:374] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.106794 18257 raft_consensus.cc:729] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.106926 18257 consensus_queue.cc:260] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.107228 18257 ts_tablet_manager.cc:1398] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:39.107424 18257 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:39.108379 18257 tablet_bootstrap.cc:654] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.108301 16938 tablet_service.cc:1467] Processing CreateTablet for tablet bcce59ab1c5947efb63e2839e30ccc72 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.108601 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcce59ab1c5947efb63e2839e30ccc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.109688 17335 tablet_service.cc:1467] Processing CreateTablet for tablet 7c25b2699528442f8e2993efbac96bbd (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.110028 17335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c25b2699528442f8e2993efbac96bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.110750 18253 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:39.111678 18253 tablet_bootstrap.cc:654] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.112347 17333 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae979c5adf44f3e8590ef06c71bf904 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.112382 16939 tablet_service.cc:1467] Processing CreateTablet for tablet 7c25b2699528442f8e2993efbac96bbd (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.112534 18260 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:39.112627 17333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae979c5adf44f3e8590ef06c71bf904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.112648 16939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c25b2699528442f8e2993efbac96bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.113008 17201 tablet_service.cc:1467] Processing CreateTablet for tablet 5ae979c5adf44f3e8590ef06c71bf904 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.113277 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae979c5adf44f3e8590ef06c71bf904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.114065 17334 tablet_service.cc:1467] Processing CreateTablet for tablet bcce59ab1c5947efb63e2839e30ccc72 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.114313 17334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcce59ab1c5947efb63e2839e30ccc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.115422 16542 tablet_service.cc:1467] Processing CreateTablet for tablet bcce59ab1c5947efb63e2839e30ccc72 (DEFAULT_TABLE table=rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.115741 16542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcce59ab1c5947efb63e2839e30ccc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.116680 18260 tablet_bootstrap.cc:654] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.117348 18256 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:39.117466 18250 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:39.118185 18260 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:39.118286 18260 ts_tablet_manager.cc:1367] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 08:00:39.118906 18250 tablet_bootstrap.cc:654] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.119091 18260 raft_consensus.cc:348] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.119230 18260 raft_consensus.cc:374] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.119262 18260 raft_consensus.cc:729] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.119428 18260 consensus_queue.cc:260] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.120930 18253 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:39.120930 18257 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:39.121050 18249 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:39.121064 18257 ts_tablet_manager.cc:1367] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240501 08:00:39.121063 18253 ts_tablet_manager.cc:1367] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 08:00:39.121165 18249 ts_tablet_manager.cc:1367] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20240501 08:00:39.121568 18257 raft_consensus.cc:348] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.121568 18253 raft_consensus.cc:348] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.121727 18253 raft_consensus.cc:374] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.121727 18257 raft_consensus.cc:374] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.121769 18257 raft_consensus.cc:729] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.121769 18253 raft_consensus.cc:729] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.121711 18249 raft_consensus.cc:348] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.121853 18249 raft_consensus.cc:374] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.121888 18249 raft_consensus.cc:729] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.121901 18253 consensus_queue.cc:260] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.121901 18257 consensus_queue.cc:260] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.122035 18249 consensus_queue.cc:260] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.122222 18253 ts_tablet_manager.cc:1398] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:39.122231 18257 ts_tablet_manager.cc:1398] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:39.122354 18249 ts_tablet_manager.cc:1398] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:39.122413 18257 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:39.122413 18253 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:39.122532 18249 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:39.123433 18253 tablet_bootstrap.cc:654] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.123432 18257 tablet_bootstrap.cc:654] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.123512 18249 tablet_bootstrap.cc:654] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.125182 18247 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:39.125334 18247 ts_tablet_manager.cc:1367] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20240501 08:00:39.125829 18247 raft_consensus.cc:348] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.125990 18247 raft_consensus.cc:374] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.126029 18247 raft_consensus.cc:729] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.126164 18247 consensus_queue.cc:260] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.126507 18247 ts_tablet_manager.cc:1398] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:39.126586 18249 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:39.126686 18249 ts_tablet_manager.cc:1367] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 08:00:39.126719 18247 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:39.127169 18249 raft_consensus.cc:348] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.127311 18249 raft_consensus.cc:374] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.127342 18249 raft_consensus.cc:729] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.127396 18260 ts_tablet_manager.cc:1398] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20240501 08:00:39.127475 18249 consensus_queue.cc:260] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.127741 18247 tablet_bootstrap.cc:654] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.127789 18249 ts_tablet_manager.cc:1398] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:39.128008 18249 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:39.129173 18249 tablet_bootstrap.cc:654] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.129542 18260 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:39.130846 18260 tablet_bootstrap.cc:654] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.131651 18256 tablet_bootstrap.cc:654] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.132301 18247 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:39.132421 18247 ts_tablet_manager.cc:1367] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 08:00:39.132611 18260 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:39.132717 18260 ts_tablet_manager.cc:1367] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:39.132915 18247 raft_consensus.cc:348] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.133040 18247 raft_consensus.cc:374] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.133062 18247 raft_consensus.cc:729] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.133170 18260 raft_consensus.cc:348] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.133220 18247 consensus_queue.cc:260] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.133282 18260 raft_consensus.cc:374] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.133314 18260 raft_consensus.cc:729] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.133515 18247 ts_tablet_manager.cc:1398] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:39.133700 18247 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:39.133848 18256 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:39.134001 18256 ts_tablet_manager.cc:1367] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240501 08:00:39.134522 18256 raft_consensus.cc:348] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.134658 18256 raft_consensus.cc:374] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.134699 18256 raft_consensus.cc:729] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.134866 18256 consensus_queue.cc:260] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.135262 18247 tablet_bootstrap.cc:654] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.135298 18256 ts_tablet_manager.cc:1398] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:39.135838 18256 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:39.136790 18256 tablet_bootstrap.cc:654] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.135484 18260 consensus_queue.cc:260] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.137740 18260 ts_tablet_manager.cc:1398] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240501 08:00:39.138233 18256 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:39.138337 18256 ts_tablet_manager.cc:1367] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:39.138803 18256 raft_consensus.cc:348] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.138916 18256 raft_consensus.cc:374] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.138944 18256 raft_consensus.cc:729] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.139071 18256 consensus_queue.cc:260] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.139369 18256 ts_tablet_manager.cc:1398] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:39.139540 18256 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:39.140411 18250 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:39.140513 18256 tablet_bootstrap.cc:654] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.140533 18250 ts_tablet_manager.cc:1367] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240501 08:00:39.141042 18250 raft_consensus.cc:348] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.141192 18250 raft_consensus.cc:374] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.141234 18250 raft_consensus.cc:729] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.141374 18250 consensus_queue.cc:260] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.141686 18250 ts_tablet_manager.cc:1398] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:39.142136 18260 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:39.145411 18260 tablet_bootstrap.cc:654] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.146664 18260 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:39.146813 18260 ts_tablet_manager.cc:1367] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 08:00:39.147261 18260 raft_consensus.cc:348] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.147387 18260 raft_consensus.cc:374] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.147418 18260 raft_consensus.cc:729] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.147545 18260 consensus_queue.cc:260] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.147854 18260 ts_tablet_manager.cc:1398] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:39.152434 18249 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:39.152434 18257 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:39.152434 18253 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:39.152575 18249 ts_tablet_manager.cc:1367] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240501 08:00:39.152572 18253 ts_tablet_manager.cc:1367] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240501 08:00:39.152571 18257 ts_tablet_manager.cc:1367] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.002s
I20240501 08:00:39.153126 18249 raft_consensus.cc:348] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.153134 18257 raft_consensus.cc:348] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.153134 18253 raft_consensus.cc:348] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.153275 18253 raft_consensus.cc:374] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.153275 18257 raft_consensus.cc:374] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.153275 18249 raft_consensus.cc:374] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.153312 18253 raft_consensus.cc:729] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.153312 18257 raft_consensus.cc:729] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.153313 18249 raft_consensus.cc:729] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.153460 18253 consensus_queue.cc:260] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.153460 18257 consensus_queue.cc:260] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.153460 18249 consensus_queue.cc:260] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.153795 18249 ts_tablet_manager.cc:1398] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:39.153798 18257 ts_tablet_manager.cc:1398] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240501 08:00:39.153797 18253 ts_tablet_manager.cc:1398] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:39.155970 18080 raft_consensus.cc:482] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:39.156134 18080 raft_consensus.cc:504] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.156816 18247 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:39.156848 18080 leader_election.cc:288] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:39.156955 18247 ts_tablet_manager.cc:1367] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240501 08:00:39.157065 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:39.157215 16957 raft_consensus.cc:2454] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 0.
I20240501 08:00:39.157423 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:39.157480 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:39.157516 18247 raft_consensus.cc:348] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.157603 17355 raft_consensus.cc:2454] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 0.
I20240501 08:00:39.157636 18247 raft_consensus.cc:374] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.157660 18247 raft_consensus.cc:729] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.157786 18247 consensus_queue.cc:260] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.157936 16694 raft_consensus.cc:2454] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 0.
I20240501 08:00:39.158286 17157 leader_election.cc:302] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7; no voters:
I20240501 08:00:39.158416 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:39.158557 16563 raft_consensus.cc:2454] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 0.
I20240501 08:00:39.159058 18080 raft_consensus.cc:2781] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:39.159060 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:39.159133 18080 raft_consensus.cc:482] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:39.159162 18080 raft_consensus.cc:3037] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.159183 17090 raft_consensus.cc:2454] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 0.
I20240501 08:00:39.159456 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
W20240501 08:00:39.159876 17156 leader_election.cc:341] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:39.160344 18080 raft_consensus.cc:504] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.160476 18247 ts_tablet_manager.cc:1398] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:39.161000 18080 leader_election.cc:288] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 1 election: Requested vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:39.161240 17931 raft_consensus.cc:482] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:39.161303 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:39.161374 17931 raft_consensus.cc:504] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.161453 17355 raft_consensus.cc:3037] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.161535 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:39.161662 16694 raft_consensus.cc:3037] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.162495 17931 leader_election.cc:288] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:39.162537 16694 raft_consensus.cc:2454] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 1.
I20240501 08:00:39.162537 17355 raft_consensus.cc:2454] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 1.
I20240501 08:00:39.162729 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:39.162907 17090 raft_consensus.cc:3037] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.163102 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcce59ab1c5947efb63e2839e30ccc72" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:39.163221 16694 raft_consensus.cc:2454] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.163414 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcce59ab1c5947efb63e2839e30ccc72" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:39.163538 17091 raft_consensus.cc:2454] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.164099 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:39.164099 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcce59ab1c5947efb63e2839e30ccc72" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:39.164166 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcce59ab1c5947efb63e2839e30ccc72" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:39.164229 16957 raft_consensus.cc:3037] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.164238 16825 raft_consensus.cc:2454] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.164301 17223 raft_consensus.cc:2454] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.164628 16498 leader_election.cc:302] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:39.164754 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcce59ab1c5947efb63e2839e30ccc72" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:39.164888 16958 raft_consensus.cc:2454] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.163955 17090 raft_consensus.cc:2454] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 1.
I20240501 08:00:39.165201 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:39.165246 16957 raft_consensus.cc:2454] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 1.
I20240501 08:00:39.165328 16563 raft_consensus.cc:3037] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.165376 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:39.165793 17159 leader_election.cc:302] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, e282199dc2fa4977b83ff07b86cb0690; no voters:
W20240501 08:00:39.165894 17156 leader_election.cc:341] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:39.166115 17931 raft_consensus.cc:2781] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:39.166182 17931 raft_consensus.cc:482] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:39.166219 17931 raft_consensus.cc:3037] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.166343 16563 raft_consensus.cc:2454] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 1.
I20240501 08:00:39.166311 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcce59ab1c5947efb63e2839e30ccc72" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:39.166450 17355 raft_consensus.cc:2454] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.167091 17931 raft_consensus.cc:504] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.167338 18080 raft_consensus.cc:2781] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:39.167627 18080 raft_consensus.cc:686] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Becoming Leader. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Running, Role: LEADER
I20240501 08:00:39.167768 17931 leader_election.cc:288] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Requested vote from peers e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:39.167876 18080 consensus_queue.cc:237] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.168347 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcce59ab1c5947efb63e2839e30ccc72" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:39.168459 17223 raft_consensus.cc:3037] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.169544 17223 raft_consensus.cc:2454] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.169499 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcce59ab1c5947efb63e2839e30ccc72" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:39.169648 16825 raft_consensus.cc:3037] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.169829 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcce59ab1c5947efb63e2839e30ccc72" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:39.169952 16957 raft_consensus.cc:3037] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.170320 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcce59ab1c5947efb63e2839e30ccc72" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:39.170445 17090 raft_consensus.cc:3037] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.170925 16957 raft_consensus.cc:2454] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.171348 17090 raft_consensus.cc:2454] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.171355 16414 catalog_manager.cc:5484] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:39.171613 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcce59ab1c5947efb63e2839e30ccc72" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:39.171669 16499 leader_election.cc:302] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:39.171736 16694 raft_consensus.cc:3037] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.172086 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcce59ab1c5947efb63e2839e30ccc72" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:39.172194 17930 raft_consensus.cc:2781] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:39.172217 17355 raft_consensus.cc:3037] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.172267 17930 raft_consensus.cc:686] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Becoming Leader. State: Replica: eca6f08d57d1453784f726199474a671, State: Running, Role: LEADER
I20240501 08:00:39.172430 17930 consensus_queue.cc:237] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.172499 18256 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:39.172591 18256 ts_tablet_manager.cc:1367] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.002s
I20240501 08:00:39.173040 18256 raft_consensus.cc:348] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.173175 18256 raft_consensus.cc:374] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.173210 18256 raft_consensus.cc:729] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.173203 17355 raft_consensus.cc:2454] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.173346 18256 consensus_queue.cc:260] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.173645 18256 ts_tablet_manager.cc:1398] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:39.174338 16694 raft_consensus.cc:2454] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.175482 16825 raft_consensus.cc:2454] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.176082 16418 catalog_manager.cc:5484] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 reported cstate change: term changed from 0 to 1, leader changed from <none> to eca6f08d57d1453784f726199474a671 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:39.273671 17831 raft_consensus.cc:482] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:39.273833 17831 raft_consensus.cc:504] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.275332 17831 leader_election.cc:288] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813)
I20240501 08:00:39.282691 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:39.282902 16957 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:39.283093 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:39.283259 17223 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:39.283955 17931 raft_consensus.cc:482] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:39.284071 17931 raft_consensus.cc:504] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.284549 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:39.284690 17090 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:39.284788 17931 leader_election.cc:288] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:39.285187 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:39.285336 16957 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.285312 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:39.285457 16825 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.285723 17220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:39.285869 17220 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.286134 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:39.286306 16694 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.286283 17291 leader_election.cc:302] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:39.286571 17831 raft_consensus.cc:2781] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:39.286634 17831 raft_consensus.cc:482] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:39.286675 17831 raft_consensus.cc:3037] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.286669 16498 leader_election.cc:302] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:39.286134 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:39.286789 16695 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:39.286861 17931 raft_consensus.cc:2781] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:39.286922 17931 raft_consensus.cc:482] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:39.286962 17931 raft_consensus.cc:3037] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.287025 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:39.287163 17090 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.287180 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:39.288161 17931 raft_consensus.cc:504] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.289239 17931 leader_election.cc:288] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Requested vote from peers 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:39.289415 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:39.289415 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:39.289556 16695 raft_consensus.cc:3037] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.289556 16957 raft_consensus.cc:3037] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.290138 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:39.290302 16825 raft_consensus.cc:3037] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.290730 16957 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.290828 16695 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.291071 16825 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.291419 16498 leader_election.cc:302] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, d5a55f6821d84c968a42f4a5c9ad07a7, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:39.291481 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:39.291764 17090 raft_consensus.cc:3037] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.291965 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:39.292023 17220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:39.292229 17220 raft_consensus.cc:3037] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.292490 17930 raft_consensus.cc:2781] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:39.292586 17930 raft_consensus.cc:686] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Becoming Leader. State: Replica: eca6f08d57d1453784f726199474a671, State: Running, Role: LEADER
I20240501 08:00:39.292757 17930 consensus_queue.cc:237] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.293115 17090 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.293339 17220 raft_consensus.cc:2454] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.295243 16418 catalog_manager.cc:5484] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 reported cstate change: term changed from 0 to 1, leader changed from <none> to eca6f08d57d1453784f726199474a671 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:39.308717 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:39.309430 17831 raft_consensus.cc:504] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.309731 17355 raft_consensus.cc:2379] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eca6f08d57d1453784f726199474a671 in current term 1: Already voted for candidate af70005d0c6c48bc9919cae6346f221b in this term.
I20240501 08:00:39.310132 17831 leader_election.cc:288] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 election: Requested vote from peers 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813)
I20240501 08:00:39.310407 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:39.310588 16695 raft_consensus.cc:2379] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate af70005d0c6c48bc9919cae6346f221b in current term 1: Already voted for candidate eca6f08d57d1453784f726199474a671 in this term.
I20240501 08:00:39.310582 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:39.310747 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:39.310935 17090 raft_consensus.cc:2379] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate af70005d0c6c48bc9919cae6346f221b in current term 1: Already voted for candidate eca6f08d57d1453784f726199474a671 in this term.
I20240501 08:00:39.311175 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:39.311225 17220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:39.311362 16957 raft_consensus.cc:2379] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate af70005d0c6c48bc9919cae6346f221b in current term 1: Already voted for candidate eca6f08d57d1453784f726199474a671 in this term.
I20240501 08:00:39.311400 17220 raft_consensus.cc:2379] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate af70005d0c6c48bc9919cae6346f221b in current term 1: Already voted for candidate eca6f08d57d1453784f726199474a671 in this term.
I20240501 08:00:39.311703 17290 leader_election.cc:302] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: af70005d0c6c48bc9919cae6346f221b; no voters: 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671
I20240501 08:00:39.311954 17831 raft_consensus.cc:2726] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240501 08:00:39.313103 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:39.313127 16827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ae979c5adf44f3e8590ef06c71bf904" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:39.313282 16825 raft_consensus.cc:2379] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate af70005d0c6c48bc9919cae6346f221b in current term 1: Already voted for candidate eca6f08d57d1453784f726199474a671 in this term.
W20240501 08:00:39.463557 18287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:39.463989 18287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:39.469235 18287 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240501 08:00:39.550246 17930 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.554744 17930 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.559197 18275 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.582589 18175 raft_consensus.cc:482] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:39.582770 18175 raft_consensus.cc:504] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.583850 18175 leader_election.cc:288] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219)
I20240501 08:00:39.584668 17220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:39.584617 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:39.585311 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:39.585512 16695 raft_consensus.cc:2379] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1160c6e72055456992b5d8bf5f3b45c0 in current term 1: Already voted for candidate 3d06b1f1890443da898fde2eb82daad3 in this term.
I20240501 08:00:39.585762 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:39.585947 16561 raft_consensus.cc:2379] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1160c6e72055456992b5d8bf5f3b45c0 in current term 1: Already voted for candidate 3d06b1f1890443da898fde2eb82daad3 in this term.
I20240501 08:00:39.586278 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:39.586472 16957 raft_consensus.cc:2379] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1160c6e72055456992b5d8bf5f3b45c0 in current term 1: Already voted for candidate 3d06b1f1890443da898fde2eb82daad3 in this term.
I20240501 08:00:39.586786 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c25b2699528442f8e2993efbac96bbd" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:39.586966 17355 raft_consensus.cc:2379] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1160c6e72055456992b5d8bf5f3b45c0 in current term 1: Already voted for candidate 3d06b1f1890443da898fde2eb82daad3 in this term.
I20240501 08:00:39.588106 16763 leader_election.cc:302] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0; no voters: 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671
I20240501 08:00:39.588747 17091 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.589339 17220 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.589386 16825 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.590378 18175 raft_consensus.cc:3037] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.590495 17355 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.590777 16957 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.590823 16695 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.592140 17931 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.592569 17931 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.592983 17931 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.593228 17931 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.593472 17931 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.593695 17931 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.595552 18275 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.598064 16827 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.600031 16694 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.600195 16958 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.600530 18080 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.600616 17353 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.602867 17220 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.604781 16958 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.605655 17930 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.609391 18275 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.611394 16561 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.611409 18324 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.612476 18324 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.612854 18322 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.618777 18080 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.591897 18175 raft_consensus.cc:2726] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 08:00:39.628780 16827 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:39.631271 18080 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:39.637130 18330 mvcc.cc:205] Tried to move back new op lower bound from 7022798600586493952 to 7022798598834995200. Current Snapshot: MvccSnapshot[applied={T|T < 7022798600586493952}]
I20240501 08:00:39.661338 18353 mvcc.cc:205] Tried to move back new op lower bound from 7022798600567402496 to 7022798598853382144. Current Snapshot: MvccSnapshot[applied={T|T < 7022798600567402496}]
I20240501 08:00:39.683120 18320 mvcc.cc:205] Tried to move back new op lower bound from 7022798600567402496 to 7022798598853382144. Current Snapshot: MvccSnapshot[applied={T|T < 7022798600567402496}]
I20240501 08:00:39.689815 18335 mvcc.cc:205] Tried to move back new op lower bound from 7022798600642736128 to 7022798599346319360. Current Snapshot: MvccSnapshot[applied={T|T < 7022798600545263616}]
I20240501 08:00:39.693701 18346 mvcc.cc:205] Tried to move back new op lower bound from 7022798600545263616 to 7022798599346319360. Current Snapshot: MvccSnapshot[applied={T|T < 7022798600545263616}]
I20240501 08:00:39.700201 18355 mvcc.cc:205] Tried to move back new op lower bound from 7022798600567402496 to 7022798598853382144. Current Snapshot: MvccSnapshot[applied={T|T < 7022798600567402496}]
I20240501 08:00:39.707348 18315 mvcc.cc:205] Tried to move back new op lower bound from 7022798600567402496 to 7022798598853382144. Current Snapshot: MvccSnapshot[applied={T|T < 7022798600567402496}]
INSERT report
rows total: 2000
time total: 269.726 ms
time per row: 0.134863 ms
I20240501 08:00:39.805722 16421 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59002:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 08:00:39.806362 16421 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_4 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 08:00:39.813657 17201 tablet_service.cc:1467] Processing CreateTablet for tablet ecd1ff8b89a84669bd8001ea731be7dc (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.813637 16939 tablet_service.cc:1467] Processing CreateTablet for tablet ecd1ff8b89a84669bd8001ea731be7dc (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.813735 17069 tablet_service.cc:1467] Processing CreateTablet for tablet ecd1ff8b89a84669bd8001ea731be7dc (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.814019 16939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd1ff8b89a84669bd8001ea731be7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.814019 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd1ff8b89a84669bd8001ea731be7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.814061 17069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd1ff8b89a84669bd8001ea731be7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.816932 16675 tablet_service.cc:1467] Processing CreateTablet for tablet ecd1ff8b89a84669bd8001ea731be7dc (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.817051 16542 tablet_service.cc:1467] Processing CreateTablet for tablet ecd1ff8b89a84669bd8001ea731be7dc (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.817297 16675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd1ff8b89a84669bd8001ea731be7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.817299 16542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd1ff8b89a84669bd8001ea731be7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.817251 16673 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd34b65ff134bf485af45d1eccd62f9 (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.817533 16673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd34b65ff134bf485af45d1eccd62f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.818125 17071 tablet_service.cc:1467] Processing CreateTablet for tablet 8bc2d601fe484082b947235237dbd42f (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.818409 17071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bc2d601fe484082b947235237dbd42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.820580 16806 tablet_service.cc:1467] Processing CreateTablet for tablet ecd1ff8b89a84669bd8001ea731be7dc (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.820914 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd1ff8b89a84669bd8001ea731be7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.820992 18401 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:39.821732 17069 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd34b65ff134bf485af45d1eccd62f9 (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.822077 17069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd34b65ff134bf485af45d1eccd62f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.822119 18401 tablet_bootstrap.cc:654] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.823217 18401 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:39.823339 18401 ts_tablet_manager.cc:1367] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:39.823680 18403 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:39.823810 16674 tablet_service.cc:1467] Processing CreateTablet for tablet 8bc2d601fe484082b947235237dbd42f (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.823885 18401 raft_consensus.cc:348] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.824079 18401 raft_consensus.cc:374] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.824119 18401 raft_consensus.cc:729] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.824179 16674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bc2d601fe484082b947235237dbd42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.824293 18401 consensus_queue.cc:260] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.825156 18403 tablet_bootstrap.cc:654] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.825166 18401 ts_tablet_manager.cc:1398] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:39.825275 18399 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:39.826288 18403 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:39.826390 18403 ts_tablet_manager.cc:1367] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:39.826829 18403 raft_consensus.cc:348] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.826944 18403 raft_consensus.cc:374] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.826964 18403 raft_consensus.cc:729] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.827108 18403 consensus_queue.cc:260] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.827476 18403 ts_tablet_manager.cc:1398] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:39.827724 17334 tablet_service.cc:1467] Processing CreateTablet for tablet ecd1ff8b89a84669bd8001ea731be7dc (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.827756 17333 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd34b65ff134bf485af45d1eccd62f9 (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.828114 17334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd1ff8b89a84669bd8001ea731be7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.828087 16939 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd34b65ff134bf485af45d1eccd62f9 (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.828203 16938 tablet_service.cc:1467] Processing CreateTablet for tablet 8bc2d601fe484082b947235237dbd42f (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.828392 16939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd34b65ff134bf485af45d1eccd62f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.829100 18399 tablet_bootstrap.cc:654] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.829429 18398 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:39.830550 18398 tablet_bootstrap.cc:654] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.830708 17335 tablet_service.cc:1467] Processing CreateTablet for tablet 8bc2d601fe484082b947235237dbd42f (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.830912 16807 tablet_service.cc:1467] Processing CreateTablet for tablet 8bc2d601fe484082b947235237dbd42f (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.831199 16807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bc2d601fe484082b947235237dbd42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.831156 17201 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd34b65ff134bf485af45d1eccd62f9 (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.831377 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd34b65ff134bf485af45d1eccd62f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.832383 18397 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:39.833608 18397 tablet_bootstrap.cc:654] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.833775 17203 tablet_service.cc:1467] Processing CreateTablet for tablet 8bc2d601fe484082b947235237dbd42f (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.834064 17203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bc2d601fe484082b947235237dbd42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.834067 18403 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:39.834374 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bc2d601fe484082b947235237dbd42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.834792 18399 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:39.834905 18399 ts_tablet_manager.cc:1367] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 08:00:39.835599 18403 tablet_bootstrap.cc:654] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.836664 16541 tablet_service.cc:1467] Processing CreateTablet for tablet 8bc2d601fe484082b947235237dbd42f (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.836665 16542 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd34b65ff134bf485af45d1eccd62f9 (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.837065 16541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bc2d601fe484082b947235237dbd42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.837507 17335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bc2d601fe484082b947235237dbd42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.839393 17333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd34b65ff134bf485af45d1eccd62f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.841192 18403 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:39.841399 18403 ts_tablet_manager.cc:1367] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.002s
I20240501 08:00:39.841967 18403 raft_consensus.cc:348] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.842114 18403 raft_consensus.cc:374] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.842147 18403 raft_consensus.cc:729] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.842273 18403 consensus_queue.cc:260] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.842586 18403 ts_tablet_manager.cc:1398] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:39.842936 18409 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:39.844054 18409 tablet_bootstrap.cc:654] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.845943 18409 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:39.846235 18409 ts_tablet_manager.cc:1367] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 08:00:39.846737 18409 raft_consensus.cc:348] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.846902 18409 raft_consensus.cc:374] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.846956 18409 raft_consensus.cc:729] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.847110 18409 consensus_queue.cc:260] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.847138 18401 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:39.847891 18409 ts_tablet_manager.cc:1398] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 08:00:39.848189 18409 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:39.848289 18401 tablet_bootstrap.cc:654] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.849026 16542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd34b65ff134bf485af45d1eccd62f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.851217 18409 tablet_bootstrap.cc:654] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.852037 18399 raft_consensus.cc:348] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.852283 18399 raft_consensus.cc:374] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.852334 18399 raft_consensus.cc:729] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.852356 16806 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd34b65ff134bf485af45d1eccd62f9 (DEFAULT_TABLE table=rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:39.852501 18399 consensus_queue.cc:260] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.852686 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd34b65ff134bf485af45d1eccd62f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:39.853444 18399 ts_tablet_manager.cc:1398] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20240501 08:00:39.853725 18399 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:39.854125 18409 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:39.854229 18409 ts_tablet_manager.cc:1367] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 08:00:39.854743 18409 raft_consensus.cc:348] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.854872 18409 raft_consensus.cc:374] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.854903 18409 raft_consensus.cc:729] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.855054 18403 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:39.855034 18409 consensus_queue.cc:260] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.855269 18399 tablet_bootstrap.cc:654] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.855332 18400 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:39.855353 18409 ts_tablet_manager.cc:1398] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:39.855536 18409 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:39.856091 18403 tablet_bootstrap.cc:654] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.856277 18400 tablet_bootstrap.cc:654] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.856652 18401 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:39.856750 18401 ts_tablet_manager.cc:1367] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 08:00:39.857286 18401 raft_consensus.cc:348] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.857429 18401 raft_consensus.cc:374] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.857461 18401 raft_consensus.cc:729] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.857753 18409 tablet_bootstrap.cc:654] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.857599 18401 consensus_queue.cc:260] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.858275 18401 ts_tablet_manager.cc:1398] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:39.858501 18401 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:39.859102 18400 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:39.859225 18400 ts_tablet_manager.cc:1367] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 08:00:39.859799 18401 tablet_bootstrap.cc:654] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.859750 18400 raft_consensus.cc:348] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.859901 18400 raft_consensus.cc:374] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.859975 18400 raft_consensus.cc:729] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.860059 18409 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:39.860149 18409 ts_tablet_manager.cc:1367] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240501 08:00:39.860123 18400 consensus_queue.cc:260] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.860427 18400 ts_tablet_manager.cc:1398] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:39.860634 18400 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:39.860625 18409 raft_consensus.cc:348] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.860754 18409 raft_consensus.cc:374] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.860790 18409 raft_consensus.cc:729] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.860924 18409 consensus_queue.cc:260] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.861233 18409 ts_tablet_manager.cc:1398] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:39.861517 18401 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:39.861517 18397 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:39.861629 18397 ts_tablet_manager.cc:1367] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20240501 08:00:39.861634 18400 tablet_bootstrap.cc:654] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.861670 18401 ts_tablet_manager.cc:1367] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 08:00:39.862162 18401 raft_consensus.cc:348] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.862162 18397 raft_consensus.cc:348] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.862313 18401 raft_consensus.cc:374] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.862356 18401 raft_consensus.cc:729] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.862457 18397 raft_consensus.cc:374] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.862532 18397 raft_consensus.cc:729] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.862493 18401 consensus_queue.cc:260] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.862804 18401 ts_tablet_manager.cc:1398] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:39.862768 18397 consensus_queue.cc:260] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.862895 18400 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:39.862985 18400 ts_tablet_manager.cc:1367] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:39.863291 18397 ts_tablet_manager.cc:1398] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:39.863459 18400 raft_consensus.cc:348] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.863564 18397 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:39.863610 18400 raft_consensus.cc:374] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.863642 18400 raft_consensus.cc:729] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.863767 18400 consensus_queue.cc:260] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.864094 18400 ts_tablet_manager.cc:1398] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:39.864275 18400 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:39.865491 18400 tablet_bootstrap.cc:654] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.867043 18400 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:39.867055 18317 raft_consensus.cc:482] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:39.867166 18400 ts_tablet_manager.cc:1367] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:39.867169 18317 raft_consensus.cc:504] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.867658 18400 raft_consensus.cc:348] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.867781 18400 raft_consensus.cc:374] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.867800 18400 raft_consensus.cc:729] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.867866 18317 leader_election.cc:288] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:39.867962 18400 consensus_queue.cc:260] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.868470 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:39.868788 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:39.868927 16694 raft_consensus.cc:2454] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.868896 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:39.869026 16826 raft_consensus.cc:2454] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.869318 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:39.869459 17353 raft_consensus.cc:2454] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 0.
I20240501 08:00:39.869530 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:39.870031 18400 ts_tablet_manager.cc:1398] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20240501 08:00:39.870110 16498 leader_election.cc:341] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813): Illegal state: must be running to vote when last-logged opid is not known
W20240501 08:00:39.870213 16499 leader_election.cc:341] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:39.870270 16499 leader_election.cc:302] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, eca6f08d57d1453784f726199474a671; no voters: 3d06b1f1890443da898fde2eb82daad3, e282199dc2fa4977b83ff07b86cb0690
I20240501 08:00:39.870455 18317 raft_consensus.cc:2781] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:39.870509 18317 raft_consensus.cc:482] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:39.870465 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:39.870555 18317 raft_consensus.cc:3037] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.871085 18397 tablet_bootstrap.cc:654] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.871611 18317 raft_consensus.cc:504] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.872278 18317 leader_election.cc:288] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Requested vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:39.872777 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:39.872928 16826 raft_consensus.cc:3037] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.873082 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:39.873214 16694 raft_consensus.cc:3037] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.873349 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
W20240501 08:00:39.873927 16498 leader_election.cc:341] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:39.874090 16826 raft_consensus.cc:2454] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.874122 16694 raft_consensus.cc:2454] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
I20240501 08:00:39.874439 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:39.874547 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:39.874668 17353 raft_consensus.cc:3037] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
W20240501 08:00:39.875170 16499 leader_election.cc:341] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:39.875669 17353 raft_consensus.cc:2454] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca6f08d57d1453784f726199474a671 in term 1.
W20240501 08:00:39.875589 16498 leader_election.cc:341] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:39.875720 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "eca6f08d57d1453784f726199474a671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:39.876142 16498 leader_election.cc:302] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, eca6f08d57d1453784f726199474a671; no voters: 3d06b1f1890443da898fde2eb82daad3, e282199dc2fa4977b83ff07b86cb0690
I20240501 08:00:39.876664 18322 raft_consensus.cc:2781] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:39.876754 18322 raft_consensus.cc:686] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Becoming Leader. State: Replica: eca6f08d57d1453784f726199474a671, State: Running, Role: LEADER
I20240501 08:00:39.876899 18322 consensus_queue.cc:237] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
W20240501 08:00:39.878938 16498 leader_election.cc:341] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:39.879536 16421 catalog_manager.cc:5484] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 reported cstate change: term changed from 0 to 1, leader changed from <none> to eca6f08d57d1453784f726199474a671 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:39.880088 18398 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:39.880218 18398 ts_tablet_manager.cc:1367] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.001s
I20240501 08:00:39.880784 18398 raft_consensus.cc:348] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.881263 18398 raft_consensus.cc:374] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.881389 18398 raft_consensus.cc:729] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.881650 18398 consensus_queue.cc:260] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.882045 18398 ts_tablet_manager.cc:1398] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 08:00:39.882652 18398 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:39.883870 18398 tablet_bootstrap.cc:654] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.884510 18399 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:39.884660 18403 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:39.884689 18399 ts_tablet_manager.cc:1367] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.000s
I20240501 08:00:39.884763 18403 ts_tablet_manager.cc:1367] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240501 08:00:39.885320 18399 raft_consensus.cc:348] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.885321 18403 raft_consensus.cc:348] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.885478 18403 raft_consensus.cc:374] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.885515 18403 raft_consensus.cc:729] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.885676 18403 consensus_queue.cc:260] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.886037 18403 ts_tablet_manager.cc:1398] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:39.886435 18399 raft_consensus.cc:374] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.886539 18399 raft_consensus.cc:729] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.886786 18399 consensus_queue.cc:260] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.888703 18399 ts_tablet_manager.cc:1398] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240501 08:00:39.889441 18399 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:39.892176 18399 tablet_bootstrap.cc:654] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.892720 18397 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:39.892699 18398 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:39.892870 18398 ts_tablet_manager.cc:1367] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20240501 08:00:39.892881 18397 ts_tablet_manager.cc:1367] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20240501 08:00:39.893585 18398 raft_consensus.cc:348] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.893728 18398 raft_consensus.cc:374] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.893760 18398 raft_consensus.cc:729] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.893687 18397 raft_consensus.cc:348] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.893931 18397 raft_consensus.cc:374] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.893978 18397 raft_consensus.cc:729] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.893987 18347 raft_consensus.cc:482] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:39.894160 18399 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:39.894081 18347 raft_consensus.cc:504] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.894147 18397 consensus_queue.cc:260] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.894270 18399 ts_tablet_manager.cc:1367] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20240501 08:00:39.894524 18397 ts_tablet_manager.cc:1398] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:39.894752 18397 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:39.894779 18347 leader_election.cc:288] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:00:39.895211 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd1ff8b89a84669bd8001ea731be7dc" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:39.895385 17353 raft_consensus.cc:2454] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:39.895344 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd1ff8b89a84669bd8001ea731be7dc" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:39.895511 16561 raft_consensus.cc:2454] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:39.895628 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd1ff8b89a84669bd8001ea731be7dc" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:39.895635 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd1ff8b89a84669bd8001ea731be7dc" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:39.895753 17091 raft_consensus.cc:2454] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:39.895773 16694 raft_consensus.cc:2454] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:39.896356 16763 leader_election.cc:302] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:39.896559 18347 raft_consensus.cc:482] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:39.896625 18334 raft_consensus.cc:2781] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:39.896688 18334 raft_consensus.cc:482] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:39.896736 18334 raft_consensus.cc:3037] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.896665 18347 raft_consensus.cc:504] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.896739 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd1ff8b89a84669bd8001ea731be7dc" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:39.896878 16958 raft_consensus.cc:2454] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:39.897439 18347 leader_election.cc:288] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:39.897835 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd1ff8b89a84669bd8001ea731be7dc" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:39.897912 18334 raft_consensus.cc:504] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.898029 17223 raft_consensus.cc:2454] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:39.898036 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:39.898160 16695 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:39.898438 18397 tablet_bootstrap.cc:654] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.898571 18334 leader_election.cc:288] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 election: Requested vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:00:39.898721 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:39.898969 16561 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:39.899371 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd1ff8b89a84669bd8001ea731be7dc" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:39.899583 17090 raft_consensus.cc:3037] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.899622 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:39.899871 18398 consensus_queue.cc:260] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.900988 17090 raft_consensus.cc:2454] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:39.901096 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd1ff8b89a84669bd8001ea731be7dc" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:39.901255 16959 raft_consensus.cc:3037] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.901289 18398 ts_tablet_manager.cc:1398] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240501 08:00:39.901515 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd1ff8b89a84669bd8001ea731be7dc" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:39.901593 18398 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:39.901643 17354 raft_consensus.cc:3037] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.902473 16959 raft_consensus.cc:2454] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:39.902617 17354 raft_consensus.cc:2454] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:39.902957 18398 tablet_bootstrap.cc:654] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:39.903192 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd1ff8b89a84669bd8001ea731be7dc" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:39.903192 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:39.903328 16561 raft_consensus.cc:3037] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.903342 16958 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:39.903827 16762 leader_election.cc:302] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:39.904070 16762 leader_election.cc:302] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, d5a55f6821d84c968a42f4a5c9ad07a7, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:39.904234 18347 raft_consensus.cc:2781] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:39.904305 18347 raft_consensus.cc:686] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Becoming Leader. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Running, Role: LEADER
I20240501 08:00:39.904354 18175 raft_consensus.cc:2781] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:39.904350 16561 raft_consensus.cc:2454] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:39.904402 18175 raft_consensus.cc:482] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:39.904435 18175 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.904472 18347 consensus_queue.cc:237] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.904778 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd1ff8b89a84669bd8001ea731be7dc" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:39.904891 16694 raft_consensus.cc:3037] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.905572 18175 raft_consensus.cc:504] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.905665 16694 raft_consensus.cc:2454] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:39.906173 18175 leader_election.cc:288] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 election: Requested vote from peers eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:39.906339 17220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd1ff8b89a84669bd8001ea731be7dc" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:39.906491 17220 raft_consensus.cc:3037] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.906605 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:39.906718 16561 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.906710 17090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:39.907178 16419 catalog_manager.cc:5484] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:39.907531 17220 raft_consensus.cc:2454] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:39.907641 16561 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:39.908349 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:39.908485 17223 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:39.909075 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:39.909212 17354 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.909247 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:39.909394 16958 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.910460 16958 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:39.910665 17354 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:39.911073 16762 leader_election.cc:302] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7, eca6f08d57d1453784f726199474a671; no voters: e282199dc2fa4977b83ff07b86cb0690
I20240501 08:00:39.911231 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:39.911317 18175 raft_consensus.cc:2781] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:39.911387 17223 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.911398 18175 raft_consensus.cc:686] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Becoming Leader. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Running, Role: LEADER
I20240501 08:00:39.911691 18175 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.912315 17223 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:39.912631 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:39.912763 16694 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.913796 16694 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:39.914394 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:39.914572 17353 raft_consensus.cc:2362] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1. Re-sending same reply.
I20240501 08:00:39.914572 16419 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:39.894822 18399 raft_consensus.cc:348] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.915259 18399 raft_consensus.cc:374] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.915362 18399 raft_consensus.cc:729] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.915544 18399 consensus_queue.cc:260] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:39.915700 17091 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:39.916026 18399 ts_tablet_manager.cc:1398] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.022s user 0.000s sys 0.002s
I20240501 08:00:39.917011 18397 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:39.917143 18397 ts_tablet_manager.cc:1367] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.002s
I20240501 08:00:39.917701 18397 raft_consensus.cc:348] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.917876 18397 raft_consensus.cc:374] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.917912 18397 raft_consensus.cc:729] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.918077 18397 consensus_queue.cc:260] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.918416 18397 ts_tablet_manager.cc:1398] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240501 08:00:39.919484 18398 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:39.919670 18398 ts_tablet_manager.cc:1367] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240501 08:00:39.920315 18398 raft_consensus.cc:348] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.920507 18398 raft_consensus.cc:374] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:39.920558 18398 raft_consensus.cc:729] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:39.920696 18398 consensus_queue.cc:260] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.921581 18398 ts_tablet_manager.cc:1398] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:39.958178 17843 raft_consensus.cc:482] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:39.958369 17843 raft_consensus.cc:504] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:39.959124 17843 leader_election.cc:288] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:39.959594 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:39.959751 17223 raft_consensus.cc:2454] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 0.
I20240501 08:00:39.959982 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:39.960029 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:39.960196 16826 raft_consensus.cc:2379] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e282199dc2fa4977b83ff07b86cb0690 in current term 1: Already voted for candidate eca6f08d57d1453784f726199474a671 in this term.
I20240501 08:00:39.960216 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:39.960364 16694 raft_consensus.cc:2379] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e282199dc2fa4977b83ff07b86cb0690 in current term 1: Already voted for candidate eca6f08d57d1453784f726199474a671 in this term.
I20240501 08:00:39.960454 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:39.960585 16958 raft_consensus.cc:2454] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e282199dc2fa4977b83ff07b86cb0690 in term 0.
I20240501 08:00:39.960848 17353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bc2d601fe484082b947235237dbd42f" candidate_uuid: "e282199dc2fa4977b83ff07b86cb0690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:39.961004 17353 raft_consensus.cc:2379] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e282199dc2fa4977b83ff07b86cb0690 in current term 1: Already voted for candidate eca6f08d57d1453784f726199474a671 in this term.
I20240501 08:00:39.961369 17026 leader_election.cc:302] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690; no voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, eca6f08d57d1453784f726199474a671
I20240501 08:00:39.961596 17843 raft_consensus.cc:3037] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:39.962877 17843 raft_consensus.cc:2726] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240501 08:00:40.083503 18428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:40.083988 18428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:40.088326 18428 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240501 08:00:40.207194 17091 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.207326 17223 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.208329 17091 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.208523 18347 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.209029 16561 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.209165 16958 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.209432 17353 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.209753 18347 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.209935 18334 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.210060 18175 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.215274 18175 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.215444 16694 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.215849 18175 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.224164 16563 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.224421 16695 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.224488 17090 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.224752 18334 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.224939 18460 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.225183 17353 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.225312 18460 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.227209 16959 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.227182 18175 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.228771 18175 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.233232 17223 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.233763 18175 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.269992 16826 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.270476 18317 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.274024 17220 raft_consensus.cc:3037] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.275334 17220 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.276835 17354 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.276953 16959 raft_consensus.cc:3037] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.278141 16959 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.280304 17090 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.282097 18318 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.282532 18318 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.282771 18476 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.284293 18322 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.289445 16694 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.290194 18317 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 227.16 ms
time per row: 0.11358 ms
I20240501 08:00:40.376579 16413 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59002:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 08:00:40.377200 16413 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_5 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 08:00:40.386569 17203 tablet_service.cc:1467] Processing CreateTablet for tablet c6841a66987744f3bc06a4b2bdbc6df2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.386833 16673 tablet_service.cc:1467] Processing CreateTablet for tablet c6841a66987744f3bc06a4b2bdbc6df2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.386978 17203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6841a66987744f3bc06a4b2bdbc6df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.387162 16673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6841a66987744f3bc06a4b2bdbc6df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.389043 16806 tablet_service.cc:1467] Processing CreateTablet for tablet c6841a66987744f3bc06a4b2bdbc6df2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.389406 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6841a66987744f3bc06a4b2bdbc6df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.389745 18398 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:40.390250 16542 tablet_service.cc:1467] Processing CreateTablet for tablet c6841a66987744f3bc06a4b2bdbc6df2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.390554 16542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6841a66987744f3bc06a4b2bdbc6df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.390851 18398 tablet_bootstrap.cc:654] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.392031 16807 tablet_service.cc:1467] Processing CreateTablet for tablet b99b55e425f045d7ba4c4133eeae9c23 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.392330 16807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99b55e425f045d7ba4c4133eeae9c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.392334 16541 tablet_service.cc:1467] Processing CreateTablet for tablet b99b55e425f045d7ba4c4133eeae9c23 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.392572 16541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99b55e425f045d7ba4c4133eeae9c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.392691 18398 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:40.392797 18398 ts_tablet_manager.cc:1367] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:40.393378 18398 raft_consensus.cc:348] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.393520 18398 raft_consensus.cc:374] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.393551 18398 raft_consensus.cc:729] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.393679 18398 consensus_queue.cc:260] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.394040 18398 ts_tablet_manager.cc:1398] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:40.395056 16543 tablet_service.cc:1467] Processing CreateTablet for tablet fa0d919844a349549b1ad6ffbe440837 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.395388 16543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0d919844a349549b1ad6ffbe440837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.395864 17069 tablet_service.cc:1467] Processing CreateTablet for tablet c6841a66987744f3bc06a4b2bdbc6df2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.396265 17069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6841a66987744f3bc06a4b2bdbc6df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.396950 17071 tablet_service.cc:1467] Processing CreateTablet for tablet b99b55e425f045d7ba4c4133eeae9c23 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.397218 17071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99b55e425f045d7ba4c4133eeae9c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.398758 18399 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:40.399755 18399 tablet_bootstrap.cc:654] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.399760 16938 tablet_service.cc:1467] Processing CreateTablet for tablet c6841a66987744f3bc06a4b2bdbc6df2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.400116 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6841a66987744f3bc06a4b2bdbc6df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.395864 17070 tablet_service.cc:1467] Processing CreateTablet for tablet fa0d919844a349549b1ad6ffbe440837 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.400547 17070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0d919844a349549b1ad6ffbe440837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.401007 18399 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:40.401093 18399 ts_tablet_manager.cc:1367] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 08:00:40.401613 18399 raft_consensus.cc:348] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.401749 18399 raft_consensus.cc:374] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.401779 18399 raft_consensus.cc:729] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.401913 18399 consensus_queue.cc:260] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.402570 18397 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:40.403637 18397 tablet_bootstrap.cc:654] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.403996 18399 ts_tablet_manager.cc:1398] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 08:00:40.404260 18399 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:40.404348 16673 tablet_service.cc:1467] Processing CreateTablet for tablet b99b55e425f045d7ba4c4133eeae9c23 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.404670 16673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99b55e425f045d7ba4c4133eeae9c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.405426 18399 tablet_bootstrap.cc:654] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.406154 16939 tablet_service.cc:1467] Processing CreateTablet for tablet fa0d919844a349549b1ad6ffbe440837 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.406280 17333 tablet_service.cc:1467] Processing CreateTablet for tablet c6841a66987744f3bc06a4b2bdbc6df2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.406551 16939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0d919844a349549b1ad6ffbe440837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.406574 17333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6841a66987744f3bc06a4b2bdbc6df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.409109 18498 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:40.409499 17335 tablet_service.cc:1467] Processing CreateTablet for tablet b99b55e425f045d7ba4c4133eeae9c23 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.409855 17335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99b55e425f045d7ba4c4133eeae9c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.410017 18498 tablet_bootstrap.cc:654] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.411146 18498 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:40.411243 18498 ts_tablet_manager.cc:1367] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:40.411684 18498 raft_consensus.cc:348] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.411804 18498 raft_consensus.cc:374] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.411824 18498 raft_consensus.cc:729] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.412034 18498 consensus_queue.cc:260] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.412319 18498 ts_tablet_manager.cc:1398] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:40.412319 17334 tablet_service.cc:1467] Processing CreateTablet for tablet fa0d919844a349549b1ad6ffbe440837 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.412739 17334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0d919844a349549b1ad6ffbe440837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.415961 17201 tablet_service.cc:1467] Processing CreateTablet for tablet fa0d919844a349549b1ad6ffbe440837 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.416111 17203 tablet_service.cc:1467] Processing CreateTablet for tablet b99b55e425f045d7ba4c4133eeae9c23 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.416309 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0d919844a349549b1ad6ffbe440837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.416570 17203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99b55e425f045d7ba4c4133eeae9c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.420297 18398 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:40.421499 18398 tablet_bootstrap.cc:654] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.422022 18397 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:40.422142 18397 ts_tablet_manager.cc:1367] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.001s
I20240501 08:00:40.422636 18397 raft_consensus.cc:348] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.422778 18397 raft_consensus.cc:374] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.422808 18397 raft_consensus.cc:729] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.422832 18398 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:40.422919 18398 ts_tablet_manager.cc:1367] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 08:00:40.422951 18397 consensus_queue.cc:260] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.423265 18397 ts_tablet_manager.cc:1398] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:40.423390 18398 raft_consensus.cc:348] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.423516 18398 raft_consensus.cc:374] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.423547 18398 raft_consensus.cc:729] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.423666 18398 consensus_queue.cc:260] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.424011 18398 ts_tablet_manager.cc:1398] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:40.431567 18498 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:40.432031 16938 tablet_service.cc:1467] Processing CreateTablet for tablet b99b55e425f045d7ba4c4133eeae9c23 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.432196 18494 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:40.432413 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99b55e425f045d7ba4c4133eeae9c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.433197 18494 tablet_bootstrap.cc:654] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.434110 18498 tablet_bootstrap.cc:654] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.434103 18492 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:40.434176 18398 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:40.435237 18492 tablet_bootstrap.cc:654] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.435446 18398 tablet_bootstrap.cc:654] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.436851 16673 tablet_service.cc:1467] Processing CreateTablet for tablet fa0d919844a349549b1ad6ffbe440837 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.437193 16673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0d919844a349549b1ad6ffbe440837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.438189 18397 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:40.439337 18397 tablet_bootstrap.cc:654] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.444187 16807 tablet_service.cc:1467] Processing CreateTablet for tablet fa0d919844a349549b1ad6ffbe440837 (DEFAULT_TABLE table=rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:40.444482 16807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0d919844a349549b1ad6ffbe440837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:40.447438 18501 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:40.448508 18501 tablet_bootstrap.cc:654] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.448905 18498 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:40.449033 18498 ts_tablet_manager.cc:1367] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240501 08:00:40.449064 18494 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:40.449177 18494 ts_tablet_manager.cc:1367] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240501 08:00:40.449580 18498 raft_consensus.cc:348] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.449728 18501 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:40.449766 18498 raft_consensus.cc:374] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.449692 18494 raft_consensus.cc:348] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.449805 18498 raft_consensus.cc:729] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.449818 18501 ts_tablet_manager.cc:1367] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:40.449836 18494 raft_consensus.cc:374] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.449868 18494 raft_consensus.cc:729] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.449993 18498 consensus_queue.cc:260] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.450400 18492 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:40.450312 18501 raft_consensus.cc:348] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.450462 18501 raft_consensus.cc:374] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.450502 18501 raft_consensus.cc:729] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.450517 18492 ts_tablet_manager.cc:1367] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240501 08:00:40.450634 18501 consensus_queue.cc:260] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.450976 18501 ts_tablet_manager.cc:1398] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:40.451084 18398 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:40.451020 18492 raft_consensus.cc:348] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.451167 18492 raft_consensus.cc:374] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.451180 18501 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:40.451195 18398 ts_tablet_manager.cc:1367] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240501 08:00:40.451197 18492 raft_consensus.cc:729] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.451377 18492 consensus_queue.cc:260] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.451607 18494 consensus_queue.cc:260] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.451718 18398 raft_consensus.cc:348] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.451859 18398 raft_consensus.cc:374] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.451895 18398 raft_consensus.cc:729] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.452051 18398 consensus_queue.cc:260] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.452304 18501 tablet_bootstrap.cc:654] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.452365 18398 ts_tablet_manager.cc:1398] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:40.452595 18492 ts_tablet_manager.cc:1398] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:40.452762 18492 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:40.453481 18498 ts_tablet_manager.cc:1398] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240501 08:00:40.453724 18498 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:40.453859 18494 ts_tablet_manager.cc:1398] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20240501 08:00:40.454049 18494 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:40.456866 18492 tablet_bootstrap.cc:654] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.457589 18399 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:40.457715 18399 ts_tablet_manager.cc:1367] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.054s user 0.003s sys 0.000s
I20240501 08:00:40.458230 18399 raft_consensus.cc:348] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.458388 18399 raft_consensus.cc:374] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.458420 18399 raft_consensus.cc:729] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.458420 18494 tablet_bootstrap.cc:654] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.458576 18399 consensus_queue.cc:260] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.459002 18399 ts_tablet_manager.cc:1398] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:40.459216 18399 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:40.460266 18399 tablet_bootstrap.cc:654] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.461799 18397 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:40.461911 18397 ts_tablet_manager.cc:1367] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20240501 08:00:40.462483 18397 raft_consensus.cc:348] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.462628 18397 raft_consensus.cc:374] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.462659 18397 raft_consensus.cc:729] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.462795 18397 consensus_queue.cc:260] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.463078 18397 ts_tablet_manager.cc:1398] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:40.463253 18397 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:40.464160 18494 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:40.464270 18494 ts_tablet_manager.cc:1367] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 08:00:40.464762 18494 raft_consensus.cc:348] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.464918 18494 raft_consensus.cc:374] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.464952 18494 raft_consensus.cc:729] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.465118 18494 consensus_queue.cc:260] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.465508 18494 ts_tablet_manager.cc:1398] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:40.465688 18494 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:40.466733 18494 tablet_bootstrap.cc:654] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.467312 18397 tablet_bootstrap.cc:654] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.468086 18492 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:40.468194 18492 ts_tablet_manager.cc:1367] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240501 08:00:40.468710 18492 raft_consensus.cc:348] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.468897 18498 tablet_bootstrap.cc:654] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.468930 18492 raft_consensus.cc:374] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.469020 18492 raft_consensus.cc:729] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.469394 18492 consensus_queue.cc:260] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.471040 18498 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:40.471184 18498 ts_tablet_manager.cc:1367] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20240501 08:00:40.471681 18399 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:40.471789 18399 ts_tablet_manager.cc:1367] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240501 08:00:40.471897 18498 raft_consensus.cc:348] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.472057 18498 raft_consensus.cc:374] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.472090 18498 raft_consensus.cc:729] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.472239 18498 consensus_queue.cc:260] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.472304 18399 raft_consensus.cc:348] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.472440 18399 raft_consensus.cc:374] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.472473 18399 raft_consensus.cc:729] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.472615 18399 consensus_queue.cc:260] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.472935 18399 ts_tablet_manager.cc:1398] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:00:40.472949 18492 ts_tablet_manager.cc:1398] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20240501 08:00:40.473183 18498 ts_tablet_manager.cc:1398] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:40.474246 18492 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:40.475386 18492 tablet_bootstrap.cc:654] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.476984 18492 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:40.477115 18492 ts_tablet_manager.cc:1367] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 08:00:40.477655 18492 raft_consensus.cc:348] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.477818 18492 raft_consensus.cc:374] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.477852 18492 raft_consensus.cc:729] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.478017 18492 consensus_queue.cc:260] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.478885 18492 ts_tablet_manager.cc:1398] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240501 08:00:40.479811 18397 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:40.479951 18397 ts_tablet_manager.cc:1367] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240501 08:00:40.480501 17831 raft_consensus.cc:482] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:40.480470 18397 raft_consensus.cc:348] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.480597 18397 raft_consensus.cc:374] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.480623 18397 raft_consensus.cc:729] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.480629 17831 raft_consensus.cc:504] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.480751 18397 consensus_queue.cc:260] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.481040 18397 ts_tablet_manager.cc:1398] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:40.481381 17831 leader_election.cc:288] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:40.481994 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0d919844a349549b1ad6ffbe440837" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:40.482151 16694 raft_consensus.cc:2454] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:40.482908 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0d919844a349549b1ad6ffbe440837" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:40.483083 17223 raft_consensus.cc:2454] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:40.483166 18494 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:40.483274 18494 ts_tablet_manager.cc:1367] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240501 08:00:40.483320 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0d919844a349549b1ad6ffbe440837" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:40.483464 16959 raft_consensus.cc:2454] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:40.483626 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0d919844a349549b1ad6ffbe440837" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:40.483791 17091 raft_consensus.cc:2454] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:40.483912 18494 raft_consensus.cc:348] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.484001 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0d919844a349549b1ad6ffbe440837" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:40.484251 17290 leader_election.cc:302] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7; no voters:
I20240501 08:00:40.484091 18494 raft_consensus.cc:374] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.484625 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0d919844a349549b1ad6ffbe440837" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:40.484679 18494 raft_consensus.cc:729] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.484854 18494 consensus_queue.cc:260] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
W20240501 08:00:40.485129 17288 leader_election.cc:341] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:40.485224 18494 ts_tablet_manager.cc:1398] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 08:00:40.484146 16561 raft_consensus.cc:2454] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:40.486268 17831 raft_consensus.cc:2781] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:40.486343 17831 raft_consensus.cc:482] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:40.486377 17831 raft_consensus.cc:3037] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.487550 17831 raft_consensus.cc:504] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.488255 17831 leader_election.cc:288] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 election: Requested vote from peers e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:40.488775 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0d919844a349549b1ad6ffbe440837" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:40.488919 16694 raft_consensus.cc:3037] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.489534 18501 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:40.489642 18501 ts_tablet_manager.cc:1367] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20240501 08:00:40.490031 16694 raft_consensus.cc:2454] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 1.
I20240501 08:00:40.490183 18501 raft_consensus.cc:348] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.490334 18501 raft_consensus.cc:374] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.490365 18501 raft_consensus.cc:729] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.490525 18501 consensus_queue.cc:260] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.490900 18501 ts_tablet_manager.cc:1398] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:40.491153 18501 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:40.491364 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0d919844a349549b1ad6ffbe440837" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:40.491499 17091 raft_consensus.cc:3037] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.492305 18501 tablet_bootstrap.cc:654] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:40.493084 17091 raft_consensus.cc:2454] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 1.
I20240501 08:00:40.493599 18501 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:40.493696 18501 ts_tablet_manager.cc:1367] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:40.494205 18501 raft_consensus.cc:348] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.494330 18501 raft_consensus.cc:374] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:40.494369 18501 raft_consensus.cc:729] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:40.494472 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0d919844a349549b1ad6ffbe440837" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:40.494529 18501 consensus_queue.cc:260] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.494596 16561 raft_consensus.cc:3037] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.494843 18501 ts_tablet_manager.cc:1398] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:40.495586 16561 raft_consensus.cc:2454] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 1.
I20240501 08:00:40.496317 17291 leader_election.cc:302] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:40.497689 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0d919844a349549b1ad6ffbe440837" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:40.497834 17223 raft_consensus.cc:3037] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.497841 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0d919844a349549b1ad6ffbe440837" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:40.497969 16959 raft_consensus.cc:3037] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.498965 17223 raft_consensus.cc:2454] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 1.
I20240501 08:00:40.498970 16959 raft_consensus.cc:2454] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 1.
I20240501 08:00:40.500360 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa0d919844a349549b1ad6ffbe440837" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:40.500506 16826 raft_consensus.cc:3037] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.501416 16826 raft_consensus.cc:2454] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 1.
I20240501 08:00:40.504056 17831 raft_consensus.cc:2781] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:40.504349 17831 raft_consensus.cc:686] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Becoming Leader. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Running, Role: LEADER
I20240501 08:00:40.504701 17831 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.509195 16414 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b reported cstate change: term changed from 0 to 1, leader changed from <none> to af70005d0c6c48bc9919cae6346f221b (127.0.99.135). New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:40.539654 18470 raft_consensus.cc:482] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:40.539829 18470 raft_consensus.cc:504] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.540565 18470 leader_election.cc:288] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:40.540874 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99b55e425f045d7ba4c4133eeae9c23" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:40.541033 16959 raft_consensus.cc:2454] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:40.541101 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99b55e425f045d7ba4c4133eeae9c23" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:40.541112 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99b55e425f045d7ba4c4133eeae9c23" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:40.541244 17354 raft_consensus.cc:2454] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:40.541260 17223 raft_consensus.cc:2454] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:40.541266 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99b55e425f045d7ba4c4133eeae9c23" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:40.541292 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99b55e425f045d7ba4c4133eeae9c23" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:40.541390 17091 raft_consensus.cc:2454] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:40.541421 16561 raft_consensus.cc:2454] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:40.541676 16763 leader_election.cc:302] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 3d06b1f1890443da898fde2eb82daad3, af70005d0c6c48bc9919cae6346f221b, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:40.541762 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99b55e425f045d7ba4c4133eeae9c23" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:40.541857 18470 raft_consensus.cc:2781] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:40.541921 18470 raft_consensus.cc:482] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:40.541965 18470 raft_consensus.cc:3037] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.541899 16694 raft_consensus.cc:2454] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:40.543262 18470 raft_consensus.cc:504] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.543946 18470 leader_election.cc:288] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 election: Requested vote from peers 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:40.544442 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99b55e425f045d7ba4c4133eeae9c23" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:40.544442 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99b55e425f045d7ba4c4133eeae9c23" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:40.544497 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99b55e425f045d7ba4c4133eeae9c23" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:40.544595 16959 raft_consensus.cc:3037] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.544591 17354 raft_consensus.cc:3037] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.544617 17091 raft_consensus.cc:3037] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.545500 17091 raft_consensus.cc:2454] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:40.545648 16959 raft_consensus.cc:2454] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:40.545648 17354 raft_consensus.cc:2454] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:40.545994 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99b55e425f045d7ba4c4133eeae9c23" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:40.546139 16561 raft_consensus.cc:3037] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.546175 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99b55e425f045d7ba4c4133eeae9c23" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:40.546269 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99b55e425f045d7ba4c4133eeae9c23" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:40.546315 17223 raft_consensus.cc:3037] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.546396 16694 raft_consensus.cc:3037] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.547142 16561 raft_consensus.cc:2454] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:40.547310 16694 raft_consensus.cc:2454] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:40.547310 17223 raft_consensus.cc:2454] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:40.547551 16760 leader_election.cc:302] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, af70005d0c6c48bc9919cae6346f221b, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:40.547775 18470 raft_consensus.cc:2781] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:40.547881 18470 raft_consensus.cc:686] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Becoming Leader. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Running, Role: LEADER
I20240501 08:00:40.548102 18470 consensus_queue.cc:237] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:40.550716 16414 catalog_manager.cc:5484] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:40.559918 18517 raft_consensus.cc:482] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:40.560099 18517 raft_consensus.cc:504] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.561233 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6841a66987744f3bc06a4b2bdbc6df2" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:40.561290 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6841a66987744f3bc06a4b2bdbc6df2" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:40.561254 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6841a66987744f3bc06a4b2bdbc6df2" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:40.561420 16561 raft_consensus.cc:2454] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:40.561446 17091 raft_consensus.cc:2454] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:40.561482 17223 raft_consensus.cc:2454] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:40.561460 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6841a66987744f3bc06a4b2bdbc6df2" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:40.561587 16959 raft_consensus.cc:2454] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:40.561718 16694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6841a66987744f3bc06a4b2bdbc6df2" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:40.561970 17290 leader_election.cc:302] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:40.562006 16694 raft_consensus.cc:2454] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:40.561985 18517 leader_election.cc:288] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:40.562180 17831 raft_consensus.cc:2781] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:40.562227 17831 raft_consensus.cc:482] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:40.562263 17831 raft_consensus.cc:3037] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.562505 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6841a66987744f3bc06a4b2bdbc6df2" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:40.562634 16826 raft_consensus.cc:2454] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 0.
I20240501 08:00:40.563252 17831 raft_consensus.cc:504] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.563849 17831 leader_election.cc:288] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 election: Requested vote from peers e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:40.564428 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6841a66987744f3bc06a4b2bdbc6df2" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:40.564457 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6841a66987744f3bc06a4b2bdbc6df2" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:40.564580 17091 raft_consensus.cc:3037] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.564580 16695 raft_consensus.cc:3037] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.565601 16695 raft_consensus.cc:2454] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 1.
I20240501 08:00:40.565632 17091 raft_consensus.cc:2454] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 1.
I20240501 08:00:40.566298 17223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6841a66987744f3bc06a4b2bdbc6df2" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:40.566454 17223 raft_consensus.cc:3037] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.566560 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6841a66987744f3bc06a4b2bdbc6df2" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:40.566684 16959 raft_consensus.cc:3037] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.566705 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6841a66987744f3bc06a4b2bdbc6df2" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:40.566857 16826 raft_consensus.cc:3037] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.567620 16959 raft_consensus.cc:2454] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 1.
I20240501 08:00:40.567610 17223 raft_consensus.cc:2454] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 1.
I20240501 08:00:40.567816 16826 raft_consensus.cc:2454] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 1.
I20240501 08:00:40.568095 17290 leader_election.cc:302] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:40.568389 17831 raft_consensus.cc:2781] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:40.568483 17831 raft_consensus.cc:686] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Becoming Leader. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Running, Role: LEADER
I20240501 08:00:40.568526 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6841a66987744f3bc06a4b2bdbc6df2" candidate_uuid: "af70005d0c6c48bc9919cae6346f221b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:40.568683 16561 raft_consensus.cc:3037] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:40.568656 17831 consensus_queue.cc:237] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:40.569509 16561 raft_consensus.cc:2454] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af70005d0c6c48bc9919cae6346f221b in term 1.
I20240501 08:00:40.571915 16414 catalog_manager.cc:5484] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b reported cstate change: term changed from 0 to 1, leader changed from <none> to af70005d0c6c48bc9919cae6346f221b (127.0.99.135). New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } }
W20240501 08:00:40.743495 18521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:40.743817 18521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:40.747416 18521 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240501 08:00:40.875010 16826 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.875994 18517 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.876677 16695 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.877727 16561 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.878224 16959 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.879086 17090 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.879916 18517 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.880637 18517 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.884192 17223 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.885131 18517 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.885583 18517 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.885847 18517 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.885936 16695 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.886656 17831 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.887234 16826 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.888506 17091 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.891690 16958 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.891469 18563 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.891707 17831 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.892367 16561 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.894145 17220 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.894475 17831 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.894848 17831 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.900388 16957 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.901088 17220 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.901124 17089 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.901265 18470 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.901553 16561 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.902988 18471 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.903339 18471 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.903450 18470 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.906781 17831 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.912909 16693 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.913343 17354 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:40.915499 18470 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:40.920235 18471 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 229.559 ms
time per row: 0.11478 ms
I20240501 08:00:41.047250 16417 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59002:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 08:00:41.048964 16417 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_6 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 08:00:41.068682 16938 tablet_service.cc:1467] Processing CreateTablet for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.069063 16938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.072392 16937 tablet_service.cc:1467] Processing CreateTablet for tablet d381bdb28e1f4374b81ab23f6882fe65 (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.072844 16937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d381bdb28e1f4374b81ab23f6882fe65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.073043 16939 tablet_service.cc:1467] Processing CreateTablet for tablet 99e2e8b8e2024846b0841efc429186ea (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.073275 16939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e2e8b8e2024846b0841efc429186ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.076128 17333 tablet_service.cc:1467] Processing CreateTablet for tablet d381bdb28e1f4374b81ab23f6882fe65 (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.076495 17333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d381bdb28e1f4374b81ab23f6882fe65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.076656 17334 tablet_service.cc:1467] Processing CreateTablet for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.076992 17334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.079412 17335 tablet_service.cc:1467] Processing CreateTablet for tablet 99e2e8b8e2024846b0841efc429186ea (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.079771 17335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e2e8b8e2024846b0841efc429186ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.080226 16674 tablet_service.cc:1467] Processing CreateTablet for tablet 99e2e8b8e2024846b0841efc429186ea (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.080422 17203 tablet_service.cc:1467] Processing CreateTablet for tablet 99e2e8b8e2024846b0841efc429186ea (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.080538 16674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e2e8b8e2024846b0841efc429186ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.080647 16675 tablet_service.cc:1467] Processing CreateTablet for tablet d381bdb28e1f4374b81ab23f6882fe65 (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.080746 17203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e2e8b8e2024846b0841efc429186ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.080850 16675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d381bdb28e1f4374b81ab23f6882fe65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.081189 16807 tablet_service.cc:1467] Processing CreateTablet for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.081477 16807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.082243 16806 tablet_service.cc:1467] Processing CreateTablet for tablet 99e2e8b8e2024846b0841efc429186ea (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.082718 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e2e8b8e2024846b0841efc429186ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.083541 18616 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:41.084264 17202 tablet_service.cc:1467] Processing CreateTablet for tablet d381bdb28e1f4374b81ab23f6882fe65 (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.084525 17202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d381bdb28e1f4374b81ab23f6882fe65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.084570 18616 tablet_bootstrap.cc:654] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.084611 16542 tablet_service.cc:1467] Processing CreateTablet for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.084908 16542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.085350 18617 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:41.085348 16805 tablet_service.cc:1467] Processing CreateTablet for tablet d381bdb28e1f4374b81ab23f6882fe65 (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.085618 16805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d381bdb28e1f4374b81ab23f6882fe65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.086388 18617 tablet_bootstrap.cc:654] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.087044 18618 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:41.080423 17201 tablet_service.cc:1467] Processing CreateTablet for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.087567 16543 tablet_service.cc:1467] Processing CreateTablet for tablet 99e2e8b8e2024846b0841efc429186ea (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.087774 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.087848 16543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e2e8b8e2024846b0841efc429186ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.088332 18618 tablet_bootstrap.cc:654] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.087884 16673 tablet_service.cc:1467] Processing CreateTablet for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.089044 16673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.089444 18620 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:41.090411 16541 tablet_service.cc:1467] Processing CreateTablet for tablet d381bdb28e1f4374b81ab23f6882fe65 (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.090688 16541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d381bdb28e1f4374b81ab23f6882fe65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.092850 17070 tablet_service.cc:1467] Processing CreateTablet for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.079412 18615 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:41.093166 17070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.094033 18614 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:41.094056 18615 tablet_bootstrap.cc:654] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.094879 18614 tablet_bootstrap.cc:654] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.095185 17069 tablet_service.cc:1467] Processing CreateTablet for tablet d381bdb28e1f4374b81ab23f6882fe65 (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.095304 18615 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:41.095402 18615 ts_tablet_manager.cc:1367] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240501 08:00:41.095430 17069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d381bdb28e1f4374b81ab23f6882fe65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.095893 18615 raft_consensus.cc:348] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.096110 18615 raft_consensus.cc:374] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.096145 18615 raft_consensus.cc:729] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.096343 18614 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:41.096323 18615 consensus_queue.cc:260] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.096433 18614 ts_tablet_manager.cc:1367] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:41.096927 18614 raft_consensus.cc:348] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.097059 18614 raft_consensus.cc:374] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.097090 18614 raft_consensus.cc:729] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.097241 18614 consensus_queue.cc:260] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.097381 18624 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:41.097581 18614 ts_tablet_manager.cc:1398] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:41.097772 18614 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:41.097910 17071 tablet_service.cc:1467] Processing CreateTablet for tablet 99e2e8b8e2024846b0841efc429186ea (DEFAULT_TABLE table=rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 08:00:41.098162 17071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e2e8b8e2024846b0841efc429186ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:41.098764 18614 tablet_bootstrap.cc:654] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.098794 18615 ts_tablet_manager.cc:1398] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:41.100064 18614 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:41.100157 18614 ts_tablet_manager.cc:1367] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:41.100342 18615 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:41.100636 18614 raft_consensus.cc:348] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.100766 18614 raft_consensus.cc:374] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.100800 18614 raft_consensus.cc:729] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.100929 18614 consensus_queue.cc:260] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.101271 18614 ts_tablet_manager.cc:1398] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:41.101459 18614 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:00:41.101738 18615 tablet_bootstrap.cc:654] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.102389 18614 tablet_bootstrap.cc:654] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.102970 18615 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:41.103058 18615 ts_tablet_manager.cc:1367] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:41.103545 18615 raft_consensus.cc:348] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.103698 18615 raft_consensus.cc:374] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.103730 18615 raft_consensus.cc:729] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.103796 18614 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7: No bootstrap required, opened a new log
I20240501 08:00:41.103878 18614 ts_tablet_manager.cc:1367] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:41.103880 18615 consensus_queue.cc:260] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.104264 18615 ts_tablet_manager.cc:1398] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:41.104457 18615 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b: Bootstrap starting.
I20240501 08:00:41.104369 18614 raft_consensus.cc:348] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.104492 18614 raft_consensus.cc:374] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.104521 18614 raft_consensus.cc:729] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.104642 18614 consensus_queue.cc:260] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.104930 18614 ts_tablet_manager.cc:1398] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:41.105481 18615 tablet_bootstrap.cc:654] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.106611 18615 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b: No bootstrap required, opened a new log
I20240501 08:00:41.106703 18615 ts_tablet_manager.cc:1367] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:41.107194 18615 raft_consensus.cc:348] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.107342 18615 raft_consensus.cc:374] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.107374 18615 raft_consensus.cc:729] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af70005d0c6c48bc9919cae6346f221b, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.107502 18615 consensus_queue.cc:260] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.107779 18624 tablet_bootstrap.cc:654] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.107836 18615 ts_tablet_manager.cc:1398] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:41.108492 18620 tablet_bootstrap.cc:654] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.109097 18616 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:41.109203 18616 ts_tablet_manager.cc:1367] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240501 08:00:41.109745 18616 raft_consensus.cc:348] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.109889 18616 raft_consensus.cc:374] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.109920 18616 raft_consensus.cc:729] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.110069 18616 consensus_queue.cc:260] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.110219 18617 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:41.110328 18617 ts_tablet_manager.cc:1367] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240501 08:00:41.110410 18616 ts_tablet_manager.cc:1398] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:41.110587 18616 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:41.110877 18617 raft_consensus.cc:348] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.111027 18617 raft_consensus.cc:374] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.111060 18617 raft_consensus.cc:729] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.111196 18617 consensus_queue.cc:260] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.111493 18617 ts_tablet_manager.cc:1398] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:41.111627 18616 tablet_bootstrap.cc:654] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.111694 18617 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:41.112968 18617 tablet_bootstrap.cc:654] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.114295 18618 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:41.114452 18618 ts_tablet_manager.cc:1367] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.000s
I20240501 08:00:41.115074 18618 raft_consensus.cc:348] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.115350 18618 raft_consensus.cc:374] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.115402 18618 raft_consensus.cc:729] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.115553 18618 consensus_queue.cc:260] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.116329 18618 ts_tablet_manager.cc:1398] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:41.116457 18624 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:41.116549 18618 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:41.116917 18620 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:41.117059 18620 ts_tablet_manager.cc:1367] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20240501 08:00:41.117254 18624 ts_tablet_manager.cc:1367] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240501 08:00:41.117722 18620 raft_consensus.cc:348] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.117897 18620 raft_consensus.cc:374] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.117933 18620 raft_consensus.cc:729] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.117831 18624 raft_consensus.cc:348] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.117952 18618 tablet_bootstrap.cc:654] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.117981 18624 raft_consensus.cc:374] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.118012 18624 raft_consensus.cc:729] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.118072 18620 consensus_queue.cc:260] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.118170 18624 consensus_queue.cc:260] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.118460 18620 ts_tablet_manager.cc:1398] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:41.118517 18624 ts_tablet_manager.cc:1398] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:41.118674 18620 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:41.118719 18624 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:41.119879 18624 tablet_bootstrap.cc:654] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.119978 18620 tablet_bootstrap.cc:654] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.121230 18616 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:41.121410 18616 ts_tablet_manager.cc:1367] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.002s
I20240501 08:00:41.121505 18620 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:41.121631 18620 ts_tablet_manager.cc:1367] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20240501 08:00:41.122057 18616 raft_consensus.cc:348] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.122210 18616 raft_consensus.cc:374] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.122238 18616 raft_consensus.cc:729] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.122169 18620 raft_consensus.cc:348] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.122308 18620 raft_consensus.cc:374] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.122340 18620 raft_consensus.cc:729] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.122370 18616 consensus_queue.cc:260] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.122468 18620 consensus_queue.cc:260] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.122794 18620 ts_tablet_manager.cc:1398] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:00:41.122797 18616 ts_tablet_manager.cc:1398] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:00:41.123003 18620 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:00:41.123037 18616 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3: Bootstrap starting.
I20240501 08:00:41.123517 18617 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:41.123622 18617 ts_tablet_manager.cc:1367] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240501 08:00:41.124106 18616 tablet_bootstrap.cc:654] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.124154 18620 tablet_bootstrap.cc:654] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.124637 18617 raft_consensus.cc:348] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.124799 18617 raft_consensus.cc:374] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.124833 18617 raft_consensus.cc:729] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.124974 18617 consensus_queue.cc:260] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.125314 18617 ts_tablet_manager.cc:1398] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:41.125546 18617 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7: Bootstrap starting.
I20240501 08:00:41.126273 18620 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671: No bootstrap required, opened a new log
I20240501 08:00:41.126372 18620 ts_tablet_manager.cc:1367] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:41.126636 18617 tablet_bootstrap.cc:654] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.126857 18620 raft_consensus.cc:348] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.127002 18620 raft_consensus.cc:374] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.127038 18620 raft_consensus.cc:729] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.127167 18620 consensus_queue.cc:260] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.127489 18620 ts_tablet_manager.cc:1398] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:41.128198 18618 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:41.128206 18624 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:41.128316 18618 ts_tablet_manager.cc:1367] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240501 08:00:41.128319 18624 ts_tablet_manager.cc:1367] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 08:00:41.128520 18616 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3: No bootstrap required, opened a new log
I20240501 08:00:41.128618 18616 ts_tablet_manager.cc:1367] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240501 08:00:41.128844 18618 raft_consensus.cc:348] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.128842 18624 raft_consensus.cc:348] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.128979 18618 raft_consensus.cc:374] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.129002 18624 raft_consensus.cc:374] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.129014 18618 raft_consensus.cc:729] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.129038 18624 raft_consensus.cc:729] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.129098 18616 raft_consensus.cc:348] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.129163 18618 consensus_queue.cc:260] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.129245 18616 raft_consensus.cc:374] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.129192 18624 consensus_queue.cc:260] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.129277 18616 raft_consensus.cc:729] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.129397 18616 consensus_queue.cc:260] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.129488 18618 ts_tablet_manager.cc:1398] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:41.129521 18624 ts_tablet_manager.cc:1398] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:41.129694 18618 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:00:41.129695 18616 ts_tablet_manager.cc:1398] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:00:41.129703 18624 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:00:41.131497 18624 tablet_bootstrap.cc:654] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.131513 18618 tablet_bootstrap.cc:654] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0: Neither blocks nor log segments found. Creating new log.
I20240501 08:00:41.133178 18617 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7: No bootstrap required, opened a new log
I20240501 08:00:41.133304 18617 ts_tablet_manager.cc:1367] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240501 08:00:41.133842 18617 raft_consensus.cc:348] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.134011 18617 raft_consensus.cc:374] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.134048 18617 raft_consensus.cc:729] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64196487691e4623b690b2869aad57f7, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.134178 18617 consensus_queue.cc:260] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.134538 18617 ts_tablet_manager.cc:1398] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:41.134784 18618 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0: No bootstrap required, opened a new log
I20240501 08:00:41.134920 18618 ts_tablet_manager.cc:1367] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240501 08:00:41.135437 18618 raft_consensus.cc:348] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.135597 18618 raft_consensus.cc:374] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.135634 18618 raft_consensus.cc:729] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.135758 18618 consensus_queue.cc:260] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.136140 18618 ts_tablet_manager.cc:1398] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:00:41.136348 18471 raft_consensus.cc:482] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:41.136446 18471 raft_consensus.cc:504] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.137069 18471 leader_election.cc:288] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219)
I20240501 08:00:41.137593 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e2e8b8e2024846b0841efc429186ea" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:41.137977 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e2e8b8e2024846b0841efc429186ea" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:41.138129 16561 raft_consensus.cc:2454] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:41.138384 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e2e8b8e2024846b0841efc429186ea" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:41.138547 16958 raft_consensus.cc:2454] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:41.138748 17220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e2e8b8e2024846b0841efc429186ea" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:41.139246 17220 raft_consensus.cc:2454] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:41.139297 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e2e8b8e2024846b0841efc429186ea" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
W20240501 08:00:41.139806 16763 leader_election.cc:341] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): Illegal state: must be running to vote when last-logged opid is not known
I20240501 08:00:41.139917 17354 raft_consensus.cc:2454] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:41.140556 16762 leader_election.cc:302] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 3d06b1f1890443da898fde2eb82daad3, d5a55f6821d84c968a42f4a5c9ad07a7, eca6f08d57d1453784f726199474a671; no voters: e282199dc2fa4977b83ff07b86cb0690
I20240501 08:00:41.140944 18471 raft_consensus.cc:2781] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:41.141041 18624 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690: No bootstrap required, opened a new log
I20240501 08:00:41.141033 18471 raft_consensus.cc:482] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:41.141098 18471 raft_consensus.cc:3037] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.141145 18624 ts_tablet_manager.cc:1367] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240501 08:00:41.141433 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e2e8b8e2024846b0841efc429186ea" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:41.141605 16695 raft_consensus.cc:2454] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 0.
I20240501 08:00:41.141680 18624 raft_consensus.cc:348] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.141826 18624 raft_consensus.cc:374] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 08:00:41.141865 18624 raft_consensus.cc:729] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: FOLLOWER
I20240501 08:00:41.141995 18624 consensus_queue.cc:260] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.142340 18624 ts_tablet_manager.cc:1398] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:41.143479 18471 raft_consensus.cc:504] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.144228 18471 leader_election.cc:288] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 election: Requested vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219)
I20240501 08:00:41.145254 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e2e8b8e2024846b0841efc429186ea" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:41.145488 16695 raft_consensus.cc:3037] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.145710 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e2e8b8e2024846b0841efc429186ea" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:41.145936 17354 raft_consensus.cc:3037] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.146665 16695 raft_consensus.cc:2454] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:41.147320 17354 raft_consensus.cc:2454] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:41.147619 17220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e2e8b8e2024846b0841efc429186ea" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:41.147796 17220 raft_consensus.cc:3037] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.149060 17220 raft_consensus.cc:2454] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:41.149267 16561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e2e8b8e2024846b0841efc429186ea" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:41.149407 16561 raft_consensus.cc:3037] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.149441 16762 leader_election.cc:302] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b; no voters:
I20240501 08:00:41.150166 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e2e8b8e2024846b0841efc429186ea" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:41.150301 17091 raft_consensus.cc:3037] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.150324 18573 raft_consensus.cc:2781] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:41.150475 16561 raft_consensus.cc:2454] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:41.150774 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e2e8b8e2024846b0841efc429186ea" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:41.150913 16958 raft_consensus.cc:3037] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.151419 17091 raft_consensus.cc:2454] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:41.151577 18573 raft_consensus.cc:686] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Becoming Leader. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Running, Role: LEADER
I20240501 08:00:41.151984 16958 raft_consensus.cc:2454] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 1.
I20240501 08:00:41.151865 18573 consensus_queue.cc:237] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.155059 16418 catalog_manager.cc:5484] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:41.268952 18610 raft_consensus.cc:482] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:41.269116 18610 raft_consensus.cc:504] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.269846 18610 leader_election.cc:288] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:41.270264 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ae6ca8ef5947f8abe9e92d69d8bb1e" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:41.270438 17091 raft_consensus.cc:2454] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 0.
I20240501 08:00:41.270412 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ae6ca8ef5947f8abe9e92d69d8bb1e" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7" is_pre_election: true
I20240501 08:00:41.270545 16695 raft_consensus.cc:2454] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 0.
I20240501 08:00:41.270648 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ae6ca8ef5947f8abe9e92d69d8bb1e" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:41.270808 16563 raft_consensus.cc:2454] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 0.
I20240501 08:00:41.271122 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ae6ca8ef5947f8abe9e92d69d8bb1e" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:41.271142 17220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ae6ca8ef5947f8abe9e92d69d8bb1e" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:41.271312 17220 raft_consensus.cc:2454] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 0.
I20240501 08:00:41.271312 16826 raft_consensus.cc:2454] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 0.
I20240501 08:00:41.270545 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ae6ca8ef5947f8abe9e92d69d8bb1e" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:41.271433 16892 leader_election.cc:302] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 64196487691e4623b690b2869aad57f7, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:41.271487 17354 raft_consensus.cc:2454] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 0.
I20240501 08:00:41.271677 18610 raft_consensus.cc:2781] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:41.271735 18610 raft_consensus.cc:482] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:41.271777 18610 raft_consensus.cc:3037] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.272692 18610 raft_consensus.cc:504] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.273308 18610 leader_election.cc:288] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [CANDIDATE]: Term 1 election: Requested vote from peers 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:41.273744 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ae6ca8ef5947f8abe9e92d69d8bb1e" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:41.273888 16563 raft_consensus.cc:3037] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.273850 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ae6ca8ef5947f8abe9e92d69d8bb1e" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:41.273965 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ae6ca8ef5947f8abe9e92d69d8bb1e" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:41.274101 16695 raft_consensus.cc:3037] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.274083 17091 raft_consensus.cc:3037] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.274542 17220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ae6ca8ef5947f8abe9e92d69d8bb1e" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:41.274735 17220 raft_consensus.cc:3037] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.275305 16563 raft_consensus.cc:2454] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 1.
I20240501 08:00:41.275465 16695 raft_consensus.cc:2454] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 1.
I20240501 08:00:41.275465 17091 raft_consensus.cc:2454] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 1.
I20240501 08:00:41.276003 16892 leader_election.cc:302] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 64196487691e4623b690b2869aad57f7, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690, eca6f08d57d1453784f726199474a671; no voters:
I20240501 08:00:41.276083 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ae6ca8ef5947f8abe9e92d69d8bb1e" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:41.276110 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ae6ca8ef5947f8abe9e92d69d8bb1e" candidate_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:41.276225 16826 raft_consensus.cc:3037] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.276250 17220 raft_consensus.cc:2454] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 1.
I20240501 08:00:41.276286 18610 raft_consensus.cc:2781] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:41.276391 18610 raft_consensus.cc:686] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Becoming Leader. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Running, Role: LEADER
I20240501 08:00:41.276557 18610 consensus_queue.cc:237] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.276810 17354 raft_consensus.cc:3037] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.277240 16826 raft_consensus.cc:2454] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 1.
I20240501 08:00:41.277706 17354 raft_consensus.cc:2454] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a55f6821d84c968a42f4a5c9ad07a7 in term 1.
I20240501 08:00:41.279304 16418 catalog_manager.cc:5484] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:41.437041 18239 raft_consensus.cc:482] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 08:00:41.437306 18239 raft_consensus.cc:504] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.438280 18239 leader_election.cc:288] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:41.438714 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d381bdb28e1f4374b81ab23f6882fe65" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" is_pre_election: true
I20240501 08:00:41.438925 16958 raft_consensus.cc:2454] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 0.
I20240501 08:00:41.438910 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d381bdb28e1f4374b81ab23f6882fe65" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b" is_pre_election: true
I20240501 08:00:41.439054 17354 raft_consensus.cc:2454] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 0.
I20240501 08:00:41.439055 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d381bdb28e1f4374b81ab23f6882fe65" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0" is_pre_election: true
I20240501 08:00:41.439193 16826 raft_consensus.cc:2454] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 0.
I20240501 08:00:41.439291 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d381bdb28e1f4374b81ab23f6882fe65" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671" is_pre_election: true
I20240501 08:00:41.439427 16563 raft_consensus.cc:2454] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 0.
I20240501 08:00:41.439581 17220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d381bdb28e1f4374b81ab23f6882fe65" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3" is_pre_election: true
I20240501 08:00:41.439622 16628 leader_election.cc:302] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7; no voters:
I20240501 08:00:41.439719 17220 raft_consensus.cc:2454] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 0.
I20240501 08:00:41.439829 18239 raft_consensus.cc:2781] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 08:00:41.439889 18239 raft_consensus.cc:482] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 08:00:41.439864 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d381bdb28e1f4374b81ab23f6882fe65" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690" is_pre_election: true
I20240501 08:00:41.439966 18239 raft_consensus.cc:3037] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.440027 17091 raft_consensus.cc:2454] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 0.
I20240501 08:00:41.441015 18239 raft_consensus.cc:504] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.441635 18239 leader_election.cc:288] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [CANDIDATE]: Term 1 election: Requested vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:41.442045 16958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d381bdb28e1f4374b81ab23f6882fe65" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:41.442046 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d381bdb28e1f4374b81ab23f6882fe65" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:41.442176 17354 raft_consensus.cc:3037] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.442176 16958 raft_consensus.cc:3037] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.443207 16958 raft_consensus.cc:2454] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 1.
I20240501 08:00:41.443207 17354 raft_consensus.cc:2454] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 1.
I20240501 08:00:41.443765 16563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d381bdb28e1f4374b81ab23f6882fe65" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:41.443765 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d381bdb28e1f4374b81ab23f6882fe65" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:41.443768 17220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d381bdb28e1f4374b81ab23f6882fe65" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:41.443907 16826 raft_consensus.cc:3037] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.443907 17220 raft_consensus.cc:3037] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.443955 16563 raft_consensus.cc:3037] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.444025 17091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d381bdb28e1f4374b81ab23f6882fe65" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:41.444141 17091 raft_consensus.cc:3037] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 0 FOLLOWER]: Advancing to term 1
I20240501 08:00:41.444970 16826 raft_consensus.cc:2454] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 1.
I20240501 08:00:41.444975 17220 raft_consensus.cc:2454] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 1.
I20240501 08:00:41.445078 17091 raft_consensus.cc:2454] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 1.
I20240501 08:00:41.445222 16563 raft_consensus.cc:2454] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 1.
I20240501 08:00:41.445454 16631 leader_election.cc:302] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:41.445768 18239 raft_consensus.cc:2781] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 08:00:41.445941 18239 raft_consensus.cc:686] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Becoming Leader. State: Replica: 64196487691e4623b690b2869aad57f7, State: Running, Role: LEADER
I20240501 08:00:41.446110 18239 consensus_queue.cc:237] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:41.448982 16413 catalog_manager.cc:5484] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64196487691e4623b690b2869aad57f7 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:41.585289 18471 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.605777 18471 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 08:00:41.627382 18647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:41.627763 18647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:41.632561 18647 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 08:00:41.637153 18471 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.643648 18576 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.647791 18573 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.649232 18576 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_6'
I20240501 08:00:41.692665 18610 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.699579 18610 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.741710 18610 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.742264 16826 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:41.742671 18239 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.745256 16957 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:41.745694 17220 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:41.745704 18239 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.746382 18239 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.746524 17354 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:41.748899 17091 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:41.748872 16563 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:41.749825 18239 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.750149 18645 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.752348 16826 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:41.752979 18645 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.754184 18607 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.764348 17223 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:41.766016 18607 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 08:00:41.779315 17354 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 08:00:41.783715 18610 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 206.746 ms
time per row: 0.103373 ms
I20240501 08:00:41.919842 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.136:34371
--local_ip_for_outbound_sockets=127.0.99.136
--tserver_master_addrs=127.0.99.190:35687
--webserver_port=36335
--webserver_interface=127.0.99.136
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 08:00:42.051864 18757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:42.052239 18757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:42.052371 18757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:42.056217 18757 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:42.056313 18757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:42.056416 18757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.136
I20240501 08:00:42.061013 18757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.136:34371
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.0.99.136
--webserver_port=36335
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.18757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.136
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:42.061996 18757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:42.063251 18757 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:42.074259 18765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:42.074637 18763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:42.075083 18762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:42.076331 18757 server_base.cc:1034] running on GCE node
I20240501 08:00:42.076874 18757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:42.077602 18757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:42.078806 18757 hybrid_clock.cc:648] HybridClock initialized: now 1714550442078778 us; error 49 us; skew 500 ppm
I20240501 08:00:42.081792 18757 webserver.cc:460] Webserver started at http://127.0.99.136:36335/ using document root <none> and password file <none>
I20240501 08:00:42.082669 18757 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:42.082764 18757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:42.087461 18757 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 08:00:42.090552 18771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:42.092190 18757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:42.092353 18757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "5aaf071b45ab469cb23321d37fad37f8"
format_stamp: "Formatted at 2024-05-01 08:00:32 on dist-test-slave-f81v"
I20240501 08:00:42.092985 18757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:42.110055 18757 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:42.110911 18757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:42.111114 18757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:42.111819 18757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:42.112934 18757 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:42.112987 18757 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:42.113034 18757 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:42.113061 18757 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:42.123875 18757 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.136:34371
I20240501 08:00:42.124490 18884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.136:34371 every 8 connection(s)
I20240501 08:00:42.125262 18757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 08:00:42.127617 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 18757
I20240501 08:00:42.128114 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.137:33839
--local_ip_for_outbound_sockets=127.0.99.137
--tserver_master_addrs=127.0.99.190:35687
--webserver_port=46789
--webserver_interface=127.0.99.137
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:42.138530 18885 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:42.138834 18885 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:42.139688 18885 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:42.141526 16419 ts_manager.cc:194] Re-registered known tserver with Master: 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371)
I20240501 08:00:42.142350 16419 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.136:39395
W20240501 08:00:42.257550 18888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:42.257891 18888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:42.258004 18888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:42.261634 18888 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:42.261729 18888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:42.261811 18888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.137
I20240501 08:00:42.265977 18888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.137:33839
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.0.99.137
--webserver_port=46789
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.18888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.137
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:42.267154 18888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:42.268566 18888 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:42.278208 18893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:42.278844 18896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:42.279426 18894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:42.280113 18888 server_base.cc:1034] running on GCE node
I20240501 08:00:42.280692 18888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:42.281491 18888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:42.282642 18888 hybrid_clock.cc:648] HybridClock initialized: now 1714550442282620 us; error 43 us; skew 500 ppm
I20240501 08:00:42.285118 18888 webserver.cc:460] Webserver started at http://127.0.99.137:46789/ using document root <none> and password file <none>
I20240501 08:00:42.285790 18888 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:42.285879 18888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:42.289749 18888 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 08:00:42.292181 18902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:42.293623 18888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:42.293771 18888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "873218458f2346348be2120e543990e2"
format_stamp: "Formatted at 2024-05-01 08:00:33 on dist-test-slave-f81v"
I20240501 08:00:42.294226 18888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:42.312726 18888 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:42.313412 18888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:42.313591 18888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:42.314160 18888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:42.315197 18888 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:42.315253 18888 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:42.315311 18888 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:42.315339 18888 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:42.324836 18888 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.137:33839
I20240501 08:00:42.324925 19015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.137:33839 every 8 connection(s)
I20240501 08:00:42.326221 18888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240501 08:00:42.336344 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 18888
I20240501 08:00:42.336829 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
/tmp/dist-test-taskrYyCEf/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/wal
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/logs
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.138:37529
--local_ip_for_outbound_sockets=127.0.99.138
--tserver_master_addrs=127.0.99.190:35687
--webserver_port=39607
--webserver_interface=127.0.99.138
--builtin_ntp_servers=127.0.99.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 08:00:42.337411 19016 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:42.337661 19016 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:42.338294 19016 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:42.339440 16419 ts_manager.cc:194] Re-registered known tserver with Master: 873218458f2346348be2120e543990e2 (127.0.99.137:33839)
I20240501 08:00:42.340148 16419 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.137:58831
W20240501 08:00:42.479152 19019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 08:00:42.479401 19019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 08:00:42.479486 19019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 08:00:42.483072 19019 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 08:00:42.483175 19019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 08:00:42.483268 19019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.138
I20240501 08:00:42.487666 19019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data
--fs_wal_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.138:37529
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--webserver_interface=127.0.99.138
--webserver_port=39607
--tserver_master_addrs=127.0.99.190:35687
--never_fsync=true
--heap_profile_path=/tmp/kudu.19019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.138
--log_dir=/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 07:43:16 UTC on 1a15035a6830
build id 69
I20240501 08:00:42.488723 19019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 08:00:42.490031 19019 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 08:00:42.499841 19025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:42.500224 19027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 08:00:42.501103 19024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 08:00:42.501899 19019 server_base.cc:1034] running on GCE node
I20240501 08:00:42.502393 19019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 08:00:42.503064 19019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 08:00:42.504235 19019 hybrid_clock.cc:648] HybridClock initialized: now 1714550442504233 us; error 54 us; skew 500 ppm
I20240501 08:00:42.506716 19019 webserver.cc:460] Webserver started at http://127.0.99.138:39607/ using document root <none> and password file <none>
I20240501 08:00:42.507344 19019 fs_manager.cc:359] Metadata directory not provided
I20240501 08:00:42.507426 19019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 08:00:42.511484 19019 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:42.513790 19033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:42.514969 19019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 08:00:42.515115 19019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data,/tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/wal
uuid: "5f8132e5a8014a6f913ee9ac3338600a"
format_stamp: "Formatted at 2024-05-01 08:00:33 on dist-test-slave-f81v"
I20240501 08:00:42.515573 19019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/wal
metadata directory: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/wal
1 data directories: /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 08:00:42.545446 19019 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 08:00:42.546159 19019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 08:00:42.546343 19019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 08:00:42.546936 19019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 08:00:42.547901 19019 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 08:00:42.547982 19019 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:42.548033 19019 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 08:00:42.548051 19019 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 08:00:42.558333 19019 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.138:37529
I20240501 08:00:42.558346 19147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.138:37529 every 8 connection(s)
I20240501 08:00:42.559588 19019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-9/data/info.pb
I20240501 08:00:42.568954 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrYyCEf/build/debug/bin/kudu as pid 19019
I20240501 08:00:42.569377 19148 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35687
I20240501 08:00:42.569648 19148 heartbeater.cc:464] Registering TS with master...
I20240501 08:00:42.570261 19148 heartbeater.cc:511] Master 127.0.99.190:35687 requested a full tablet report, sending...
I20240501 08:00:42.571609 16419 ts_manager.cc:194] Re-registered known tserver with Master: 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529)
I20240501 08:00:42.572252 16419 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.138:43743
I20240501 08:00:42.818012 16807 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:42.820053 17201 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:42.822749 16674 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:42.837272 18818 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:42.848650 17335 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:42.859460 16541 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:42.861449 18950 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:42.878078 19076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:42.901875 16939 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:42.903307 17071 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:43.023828 16807 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:43.028070 19076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:43.030050 18950 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:43.032804 16939 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:43.035418 17071 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:43.043361 16674 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:43.045244 17201 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:43.052363 16541 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:43.057915 18818 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:43.068117 17335 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:43.145592 18885 heartbeater.cc:503] Master 127.0.99.190:35687 was elected leader, sending a full tablet report...
I20240501 08:00:43.251313 18818 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:43.255249 16807 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:43.258571 19076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:43.261703 17335 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:43.272672 17201 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:43.285432 16674 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:43.289028 18950 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:43.291836 16541 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:43.293040 16939 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:43.333127 17071 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:43.342870 19016 heartbeater.cc:503] Master 127.0.99.190:35687 was elected leader, sending a full tablet report...
I20240501 08:00:43.376101 16562 consensus_queue.cc:237] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:43.378120 17354 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.378194 17223 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.378239 16958 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.378675 16693 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.378695 18476 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.379076 18476 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.379295 19249 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.379521 16827 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.380052 17089 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.380452 19297 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.380636 19249 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.380998 19249 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.383472 19298 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.383910 16827 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.384830 17089 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.386469 16958 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.388244 16413 catalog_manager.cc:5484] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.388329 16693 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.392138 17354 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.397568 16562 consensus_queue.cc:237] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:43.400575 16693 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.401052 16958 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.401100 16827 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.401152 17089 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.401477 19249 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.401659 19298 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.401906 18476 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.401969 17354 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.402243 18476 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.402487 19299 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.407382 17221 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 08:00:43.407886 16496 consensus_peers.cc:485] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bcce59ab1c5947efb63e2839e30ccc72. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:43.408236 19249 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.409111 17221 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.410076 19249 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.412523 16694 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.412951 17354 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.413959 16405 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet bcce59ab1c5947efb63e2839e30ccc72 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:43.414427 17221 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.414736 16415 catalog_manager.cc:5484] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:43.416369 16958 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.417246 17089 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.418077 16957 consensus_queue.cc:237] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:43.420909 17354 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.421355 18714 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.421432 16826 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.421655 16562 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.422266 18714 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.422339 16693 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.422807 18714 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.424297 17221 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.424566 16827 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.424752 18714 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.424963 17089 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.425017 19318 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.425891 19320 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.426717 18714 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.430269 16693 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.430269 16827 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.431330 17354 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.434335 17221 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.434531 16413 catalog_manager.cc:5484] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
W20240501 08:00:43.439769 19317 consensus_queue.cc:1159] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "64196487691e4623b690b2869aad57f7" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
W20240501 08:00:43.439911 19317 consensus_queue.cc:1159] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1160c6e72055456992b5d8bf5f3b45c0" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
W20240501 08:00:43.439992 19317 consensus_queue.cc:1159] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "af70005d0c6c48bc9919cae6346f221b" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
I20240501 08:00:43.445837 16957 consensus_queue.cc:237] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:43.448490 16693 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.448793 16562 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.449128 16827 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.449290 19317 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.449613 19317 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.449703 17089 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.449806 17354 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.450415 19319 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.450570 16562 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.450759 17221 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.451561 19319 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.452777 18714 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.453152 18714 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.454463 17089 raft_consensus.cc:858] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }.
Pending config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }:
New config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
W20240501 08:00:43.454547 17089 raft_consensus.cc:1565] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 5' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }.
Pending config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }:
New config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:43.455188 17089 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.457928 19318 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.460224 16407 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:43.460397 17220 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
W20240501 08:00:43.461627 16893 consensus_peers.cc:485] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c8ae6ca8ef5947f8abe9e92d69d8bb1e. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:43.462555 16562 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.463562 17354 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.464279 16418 catalog_manager.cc:5484] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.464543 16693 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.465034 16827 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:43.468114 16957 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.468379 16693 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.468492 17353 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.468626 17091 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.468863 19101 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.469166 17220 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.469115 18744 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.469482 19101 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.469803 19101 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.469879 16562 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.470325 19101 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.470459 18744 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.474260 16827 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.477954 17090 raft_consensus.cc:1203] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 08:00:43.480875 17090 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.484897 19341 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.486418 16957 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.487006 16693 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.489132 17354 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.489292 17090 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.490360 16418 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.496474 17220 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.500114 16563 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.503084 16563 consensus_queue.cc:237] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:43.505017 16827 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:43.507436 17220 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.507565 17354 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.508184 16957 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.508121 19297 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.508541 19299 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.508932 19299 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.510147 16694 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.510280 16827 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.510562 19298 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.510665 17353 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.510890 17220 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.511332 19342 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.511757 19342 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.511889 16957 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.512339 16693 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.512462 16563 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.512820 19298 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.513162 17089 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.513303 18744 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.513566 19298 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.515694 18744 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.516196 19298 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.516350 19342 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.517294 17090 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.518009 19340 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.518250 19341 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.519029 16694 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.519778 16827 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.520100 16957 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.522485 16418 catalog_manager.cc:5484] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:43.522506 17090 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.525555 16563 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.526202 16562 consensus_queue.cc:237] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
W20240501 08:00:43.528688 16761 consensus_peers.cc:485] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6fd34b65ff134bf485af45d1eccd62f9. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:43.529208 16417 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.530571 17220 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.531152 17355 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
W20240501 08:00:43.534164 16497 consensus_peers.cc:485] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b8a14c2447e45c5b148ff762bb55c00. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:43.534015 16693 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.537580 16405 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 6fd34b65ff134bf485af45d1eccd62f9 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:43.539149 17220 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.539189 16957 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.541283 16827 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.542225 17221 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.542245 16957 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.542245 19297 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.542635 19297 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.542941 19249 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.543229 16693 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.543905 19249 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.546921 19299 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.547422 17353 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.547641 16827 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.548039 17090 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.549196 19297 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.549571 17353 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:43.550401 16405 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4b8a14c2447e45c5b148ff762bb55c00 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:43.551000 19297 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.551080 16957 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.551080 16693 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.552227 16958 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.552227 17220 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.553973 17221 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.554117 16419 catalog_manager.cc:5484] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.554276 19368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.556823 19369 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.557224 17090 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.557360 17089 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.558195 16827 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.558238 16562 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.558845 19248 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.559034 19368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.559110 17089 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.559391 19368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.562252 17353 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.563747 17353 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.569314 16693 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.569779 19248 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.572127 19369 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.572746 16693 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.574537 16562 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.574559 16957 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.575098 19148 heartbeater.cc:503] Master 127.0.99.190:35687 was elected leader, sending a full tablet report...
I20240501 08:00:43.577425 16827 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.577724 17220 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.579241 16413 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.581153 17090 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.585143 17353 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:43.587579 16957 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.588339 16693 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.588629 17220 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.589072 16562 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.589144 19369 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.589190 16827 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.589521 19248 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.589725 19372 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.590101 19369 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.590751 19368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.592288 17090 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.592922 19248 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.593827 19370 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.595098 16827 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.595331 17090 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.598951 16417 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.599787 16407 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet fa0d919844a349549b1ad6ffbe440837 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:43.600430 16562 consensus_queue.cc:237] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:43.600533 16563 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.600858 16693 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.602514 17220 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.602761 16958 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.603391 17221 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.604878 16827 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.605149 17090 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.605898 19297 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.606303 19297 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.606499 16693 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.606578 19298 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.607811 19298 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.609100 19298 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.611167 17353 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.611594 19298 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.612378 16957 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.615597 19298 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.616791 16693 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
W20240501 08:00:43.619007 17288 consensus_peers.cc:485] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fa0d919844a349549b1ad6ffbe440837. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:43.618881 17090 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.619349 16827 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.621321 16413 catalog_manager.cc:5484] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.623847 17220 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.624059 17354 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.624315 16958 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.631291 16563 consensus_queue.cc:237] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:43.634042 16827 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.634166 17090 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.634666 18476 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 08:00:43.634919 16496 consensus_peers.cc:485] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: addb23c607754d0aabee4c934f390251. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:43.635021 19395 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.636685 16693 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.637048 16957 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.637092 17220 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.637126 17354 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.637147 19249 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.637416 19297 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.637574 19299 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.637866 19299 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.640205 19249 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.641247 16958 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.641973 16405 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet addb23c607754d0aabee4c934f390251 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:43.642647 17090 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.642774 16827 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.643225 16693 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.644778 17220 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.646255 16417 catalog_manager.cc:5484] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.648428 17354 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:43.658978 17220 consensus_queue.cc:237] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:43.661162 16693 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.661477 17354 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.661552 16827 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.662000 19403 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.662293 18643 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.662442 16563 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.662633 18643 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.662873 19405 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.664245 17090 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.664647 19405 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.664808 16958 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.665223 19405 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.667124 19402 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.667574 17090 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.668830 16827 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.669739 17354 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.670368 16416 catalog_manager.cc:5484] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:43.672219 16563 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.677258 17220 consensus_queue.cc:237] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:43.682632 16563 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.685264 19403 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.685369 16827 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.686540 19418 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.687522 17089 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.688242 19418 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.689627 16694 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.690310 17354 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.691103 19416 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.691892 19403 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.692996 16957 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.693562 17090 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.693743 16694 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.695245 16958 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:43.695339 16694 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.697935 16827 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.698165 17220 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.698432 16693 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.698663 19320 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.698901 19327 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.698911 19418 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.699375 19318 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.701040 17090 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.701498 19318 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 08:00:43.702400 17157 consensus_peers.cc:485] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c25b2699528442f8e2993efbac96bbd. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:43.702610 16419 catalog_manager.cc:5484] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.702749 19402 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.703397 16407 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 7c25b2699528442f8e2993efbac96bbd with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:43.703948 16826 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.704092 17353 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.706593 19327 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.707427 17220 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.707427 16562 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.710064 16826 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.710443 16417 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:43.710405 16693 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.711364 16563 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.711884 19320 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.713583 16563 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.715421 16958 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:43.717443 17354 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.718590 16958 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.719918 19320 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.721272 16957 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:43.723253 17354 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.723985 16693 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.724556 16563 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.724556 17090 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.725244 16826 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 08:00:43.725340 16893 consensus_peers.cc:485] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5ecdf2c2cdb4487b0e6315f63e1b1b8. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:43.725481 19317 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.725800 19317 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.726742 17090 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.727141 19329 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.727490 19329 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.728672 17220 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.729089 19317 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.730970 19329 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.731433 17354 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.731788 17220 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.732298 19317 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.733366 16693 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.734825 16418 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:43.735201 16407 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:43.735752 16826 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.737277 17354 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.737980 16563 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.740725 17090 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:43.748204 16563 consensus_queue.cc:237] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:43.749826 17089 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.749935 17220 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.750267 19394 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.750376 16957 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.750409 17354 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.750558 19297 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.751194 19394 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.751425 19297 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.752746 16693 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.752746 16826 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:43.753155 19394 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.754092 19394 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.755645 18476 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.756207 17354 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.756269 17220 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.756701 16826 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.759493 16418 catalog_manager.cc:5484] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.759894 17090 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.760942 16693 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.764375 16563 consensus_queue.cc:237] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:43.766580 16958 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.766615 17354 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.767220 19298 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:43.767393 17220 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.769006 16957 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:43.771186 16826 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 08:00:43.771553 16497 consensus_peers.cc:485] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ae979c5adf44f3e8590ef06c71bf904. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:43.771631 19394 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.772533 19394 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.772840 19394 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.774207 17090 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.774690 19298 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.775379 16693 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:43.775854 19298 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:43.777112 19395 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.779760 16693 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.779772 16826 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.780992 17090 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.783061 16405 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 5ae979c5adf44f3e8590ef06c71bf904 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:43.783617 16418 catalog_manager.cc:5484] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.785218 16957 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.785508 17220 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.785650 17354 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:43.944473 19488 ts_tablet_manager.cc:902] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:00:43.945884 19488 tablet_copy_client.cc:308] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:43741
I20240501 08:00:43.951346 16583 tablet_copy_service.cc:140] P eca6f08d57d1453784f726199474a671: Received BeginTabletCopySession request for tablet bcce59ab1c5947efb63e2839e30ccc72 from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:51849)
I20240501 08:00:43.951524 16583 tablet_copy_service.cc:161] P eca6f08d57d1453784f726199474a671: Beginning new tablet copy session on tablet bcce59ab1c5947efb63e2839e30ccc72 from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:51849: session id = 873218458f2346348be2120e543990e2-bcce59ab1c5947efb63e2839e30ccc72
I20240501 08:00:43.953325 16583 tablet_copy_source_session.cc:205] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:43.957115 19488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcce59ab1c5947efb63e2839e30ccc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:43.963959 19488 tablet_copy_client.cc:783] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:43.964568 19488 tablet_copy_client.cc:647] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:43.965883 19494 ts_tablet_manager.cc:902] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:43.967104 19494 tablet_copy_client.cc:308] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:43779
I20240501 08:00:43.967463 19488 tablet_copy_client.cc:523] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:43.969969 19488 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:00:43.971659 16847 tablet_copy_service.cc:140] P 1160c6e72055456992b5d8bf5f3b45c0: Received BeginTabletCopySession request for tablet 6fd34b65ff134bf485af45d1eccd62f9 from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:45789)
I20240501 08:00:43.971822 16847 tablet_copy_service.cc:161] P 1160c6e72055456992b5d8bf5f3b45c0: Beginning new tablet copy session on tablet 6fd34b65ff134bf485af45d1eccd62f9 from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:45789: session id = 5f8132e5a8014a6f913ee9ac3338600a-6fd34b65ff134bf485af45d1eccd62f9
I20240501 08:00:43.973304 16847 tablet_copy_source_session.cc:205] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:43.973876 19500 ts_tablet_manager.cc:902] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987)
I20240501 08:00:43.974797 19500 tablet_copy_client.cc:308] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:41987
I20240501 08:00:43.977599 19494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd34b65ff134bf485af45d1eccd62f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:43.979482 19488 log.cc:824] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:43.980441 16979 tablet_copy_service.cc:140] P d5a55f6821d84c968a42f4a5c9ad07a7: Received BeginTabletCopySession request for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:50425)
I20240501 08:00:43.980609 16979 tablet_copy_service.cc:161] P d5a55f6821d84c968a42f4a5c9ad07a7: Beginning new tablet copy session on tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:50425: session id = 5aaf071b45ab469cb23321d37fad37f8-c8ae6ca8ef5947f8abe9e92d69d8bb1e
I20240501 08:00:43.982102 16979 tablet_copy_source_session.cc:205] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:43.985762 19494 tablet_copy_client.cc:783] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:43.986121 19494 tablet_copy_client.cc:647] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:43.986367 19500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:43.989567 19494 tablet_copy_client.cc:523] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:43.992410 19494 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:00:43.996613 19500 tablet_copy_client.cc:783] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:43.996963 19500 tablet_copy_client.cc:647] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:43.999683 19500 tablet_copy_client.cc:523] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:44.002061 19494 log.cc:824] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:44.002398 19500 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:00:44.011458 19500 log.cc:824] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8: Log is configured to *not* fsync() on all Append() calls
I20240501 08:00:44.015187 19488 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:44.015664 19488 tablet_bootstrap.cc:492] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:00:44.016069 19488 ts_tablet_manager.cc:1367] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.046s user 0.032s sys 0.007s
I20240501 08:00:44.018762 19488 raft_consensus.cc:348] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:44.019080 19488 raft_consensus.cc:729] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:00:44.019665 19488 consensus_queue.cc:260] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:44.021260 19488 ts_tablet_manager.cc:1398] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20240501 08:00:44.022215 16583 tablet_copy_service.cc:342] P eca6f08d57d1453784f726199474a671: Request end of tablet copy session 873218458f2346348be2120e543990e2-bcce59ab1c5947efb63e2839e30ccc72 received from {username='slave'} at 127.0.99.137:51849
I20240501 08:00:44.022321 16583 tablet_copy_service.cc:433] P eca6f08d57d1453784f726199474a671: ending tablet copy session 873218458f2346348be2120e543990e2-bcce59ab1c5947efb63e2839e30ccc72 on tablet bcce59ab1c5947efb63e2839e30ccc72 with peer 873218458f2346348be2120e543990e2
I20240501 08:00:44.033603 19494 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:44.034318 19494 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:00:44.034677 19488 ts_tablet_manager.cc:902] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:44.034797 19494 ts_tablet_manager.cc:1367] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.043s user 0.035s sys 0.004s
I20240501 08:00:44.035241 19488 tablet_copy_client.cc:308] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:39065
I20240501 08:00:44.038354 19494 raft_consensus.cc:348] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:44.039067 19494 raft_consensus.cc:729] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:00:44.039610 19494 consensus_queue.cc:260] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:44.040884 19494 ts_tablet_manager.cc:1398] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20240501 08:00:44.041286 17375 tablet_copy_service.cc:140] P af70005d0c6c48bc9919cae6346f221b: Received BeginTabletCopySession request for tablet fa0d919844a349549b1ad6ffbe440837 from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:53219)
I20240501 08:00:44.041452 17375 tablet_copy_service.cc:161] P af70005d0c6c48bc9919cae6346f221b: Beginning new tablet copy session on tablet fa0d919844a349549b1ad6ffbe440837 from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:53219: session id = 873218458f2346348be2120e543990e2-fa0d919844a349549b1ad6ffbe440837
I20240501 08:00:44.041822 16847 tablet_copy_service.cc:342] P 1160c6e72055456992b5d8bf5f3b45c0: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-6fd34b65ff134bf485af45d1eccd62f9 received from {username='slave'} at 127.0.99.138:45789
I20240501 08:00:44.041936 16847 tablet_copy_service.cc:433] P 1160c6e72055456992b5d8bf5f3b45c0: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-6fd34b65ff134bf485af45d1eccd62f9 on tablet 6fd34b65ff134bf485af45d1eccd62f9 with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:00:44.043017 17375 tablet_copy_source_session.cc:205] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:44.044960 19488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0d919844a349549b1ad6ffbe440837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:44.048200 19514 ts_tablet_manager.cc:902] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:00:44.048453 19488 tablet_copy_client.cc:783] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:44.048782 19488 tablet_copy_client.cc:647] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:44.048925 19514 tablet_copy_client.cc:308] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:43741
I20240501 08:00:44.049793 16583 tablet_copy_service.cc:140] P eca6f08d57d1453784f726199474a671: Received BeginTabletCopySession request for tablet addb23c607754d0aabee4c934f390251 from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:51849)
I20240501 08:00:44.050024 16583 tablet_copy_service.cc:161] P eca6f08d57d1453784f726199474a671: Beginning new tablet copy session on tablet addb23c607754d0aabee4c934f390251 from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:51849: session id = 873218458f2346348be2120e543990e2-addb23c607754d0aabee4c934f390251
I20240501 08:00:44.051115 19488 tablet_copy_client.cc:523] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:44.051493 16583 tablet_copy_source_session.cc:205] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:44.053203 19514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addb23c607754d0aabee4c934f390251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:44.053225 19488 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:00:44.059542 19514 tablet_copy_client.cc:783] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:44.059839 19514 tablet_copy_client.cc:647] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:44.061102 19514 tablet_copy_client.cc:523] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:44.061558 19500 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:44.061976 19500 tablet_bootstrap.cc:492] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:00:44.062333 19500 ts_tablet_manager.cc:1367] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.060s user 0.035s sys 0.005s
I20240501 08:00:44.064579 19514 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:00:44.066015 19500 raft_consensus.cc:348] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:44.066385 19500 raft_consensus.cc:729] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:00:44.066820 19500 consensus_queue.cc:260] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:44.068259 19500 ts_tablet_manager.cc:1398] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20240501 08:00:44.070080 16979 tablet_copy_service.cc:342] P d5a55f6821d84c968a42f4a5c9ad07a7: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-c8ae6ca8ef5947f8abe9e92d69d8bb1e received from {username='slave'} at 127.0.99.136:50425
I20240501 08:00:44.070194 16979 tablet_copy_service.cc:433] P d5a55f6821d84c968a42f4a5c9ad07a7: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-c8ae6ca8ef5947f8abe9e92d69d8bb1e on tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:00:44.086843 19488 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:44.087184 19488 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:00:44.087471 19488 ts_tablet_manager.cc:1367] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.034s user 0.026s sys 0.000s
I20240501 08:00:44.088181 19488 raft_consensus.cc:348] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:44.088351 19488 raft_consensus.cc:729] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:00:44.088835 19488 consensus_queue.cc:260] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:44.089181 19488 ts_tablet_manager.cc:1398] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20240501 08:00:44.090029 17375 tablet_copy_service.cc:342] P af70005d0c6c48bc9919cae6346f221b: Request end of tablet copy session 873218458f2346348be2120e543990e2-fa0d919844a349549b1ad6ffbe440837 received from {username='slave'} at 127.0.99.137:53219
I20240501 08:00:44.090127 17375 tablet_copy_service.cc:433] P af70005d0c6c48bc9919cae6346f221b: ending tablet copy session 873218458f2346348be2120e543990e2-fa0d919844a349549b1ad6ffbe440837 on tablet fa0d919844a349549b1ad6ffbe440837 with peer 873218458f2346348be2120e543990e2
I20240501 08:00:44.096392 19514 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:44.096825 19514 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:00:44.097054 19514 ts_tablet_manager.cc:1367] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.033s user 0.029s sys 0.000s
I20240501 08:00:44.097615 19514 raft_consensus.cc:348] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:44.097805 19514 raft_consensus.cc:729] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:00:44.097915 19514 consensus_queue.cc:260] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:44.098325 19514 ts_tablet_manager.cc:1398] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:44.098903 16583 tablet_copy_service.cc:342] P eca6f08d57d1453784f726199474a671: Request end of tablet copy session 873218458f2346348be2120e543990e2-addb23c607754d0aabee4c934f390251 received from {username='slave'} at 127.0.99.137:51849
I20240501 08:00:44.098992 16583 tablet_copy_service.cc:433] P eca6f08d57d1453784f726199474a671: ending tablet copy session 873218458f2346348be2120e543990e2-addb23c607754d0aabee4c934f390251 on tablet addb23c607754d0aabee4c934f390251 with peer 873218458f2346348be2120e543990e2
I20240501 08:00:44.146582 19494 ts_tablet_manager.cc:902] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:00:44.147467 19494 tablet_copy_client.cc:308] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:43741
I20240501 08:00:44.153417 16583 tablet_copy_service.cc:140] P eca6f08d57d1453784f726199474a671: Received BeginTabletCopySession request for tablet 4b8a14c2447e45c5b148ff762bb55c00 from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:34739)
I20240501 08:00:44.153553 16583 tablet_copy_service.cc:161] P eca6f08d57d1453784f726199474a671: Beginning new tablet copy session on tablet 4b8a14c2447e45c5b148ff762bb55c00 from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:34739: session id = 5f8132e5a8014a6f913ee9ac3338600a-4b8a14c2447e45c5b148ff762bb55c00
I20240501 08:00:44.154570 16583 tablet_copy_source_session.cc:205] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:44.155539 19494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8a14c2447e45c5b148ff762bb55c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:44.158211 19494 tablet_copy_client.cc:783] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:44.160197 19494 tablet_copy_client.cc:647] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:44.161643 19494 tablet_copy_client.cc:523] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:44.165807 19494 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:00:44.187150 19494 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:44.187453 19494 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:00:44.187644 19494 ts_tablet_manager.cc:1367] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.022s user 0.018s sys 0.000s
I20240501 08:00:44.188664 19494 raft_consensus.cc:348] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:44.188858 19494 raft_consensus.cc:729] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:00:44.188987 19494 consensus_queue.cc:260] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:44.189471 19494 ts_tablet_manager.cc:1398] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.002s user 0.002s sys 0.004s
I20240501 08:00:44.190306 16583 tablet_copy_service.cc:342] P eca6f08d57d1453784f726199474a671: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-4b8a14c2447e45c5b148ff762bb55c00 received from {username='slave'} at 127.0.99.138:34739
I20240501 08:00:44.190415 16583 tablet_copy_service.cc:433] P eca6f08d57d1453784f726199474a671: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-4b8a14c2447e45c5b148ff762bb55c00 on tablet 4b8a14c2447e45c5b148ff762bb55c00 with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:00:44.230034 19500 ts_tablet_manager.cc:902] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987)
I20240501 08:00:44.230525 19500 tablet_copy_client.cc:308] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:41987
I20240501 08:00:44.231138 16979 tablet_copy_service.cc:140] P d5a55f6821d84c968a42f4a5c9ad07a7: Received BeginTabletCopySession request for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:50425)
I20240501 08:00:44.231422 16979 tablet_copy_service.cc:161] P d5a55f6821d84c968a42f4a5c9ad07a7: Beginning new tablet copy session on tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:50425: session id = 5aaf071b45ab469cb23321d37fad37f8-d5ecdf2c2cdb4487b0e6315f63e1b1b8
I20240501 08:00:44.232620 16979 tablet_copy_source_session.cc:205] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:44.234377 19500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:44.236380 19500 tablet_copy_client.cc:783] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:44.236557 19500 tablet_copy_client.cc:647] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:44.238250 19500 tablet_copy_client.cc:523] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:44.240023 19500 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:00:44.244923 19494 ts_tablet_manager.cc:902] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:00:44.245522 19494 tablet_copy_client.cc:308] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:43741
I20240501 08:00:44.246052 16583 tablet_copy_service.cc:140] P eca6f08d57d1453784f726199474a671: Received BeginTabletCopySession request for tablet 5ae979c5adf44f3e8590ef06c71bf904 from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:34739)
I20240501 08:00:44.246163 16583 tablet_copy_service.cc:161] P eca6f08d57d1453784f726199474a671: Beginning new tablet copy session on tablet 5ae979c5adf44f3e8590ef06c71bf904 from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:34739: session id = 5f8132e5a8014a6f913ee9ac3338600a-5ae979c5adf44f3e8590ef06c71bf904
I20240501 08:00:44.246949 16583 tablet_copy_source_session.cc:205] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:44.248003 19494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae979c5adf44f3e8590ef06c71bf904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:44.249890 19494 tablet_copy_client.cc:783] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:44.250059 19494 tablet_copy_client.cc:647] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:44.251322 19494 tablet_copy_client.cc:523] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:44.252574 19494 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:00:44.267504 19500 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:44.267853 19500 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:00:44.268069 19500 ts_tablet_manager.cc:1367] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.028s user 0.022s sys 0.004s
I20240501 08:00:44.268743 19500 raft_consensus.cc:348] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:44.268914 19500 raft_consensus.cc:729] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:00:44.269022 19500 consensus_queue.cc:260] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:44.269367 19500 ts_tablet_manager.cc:1398] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:44.270197 16979 tablet_copy_service.cc:342] P d5a55f6821d84c968a42f4a5c9ad07a7: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-d5ecdf2c2cdb4487b0e6315f63e1b1b8 received from {username='slave'} at 127.0.99.136:50425
I20240501 08:00:44.270323 16979 tablet_copy_service.cc:433] P d5a55f6821d84c968a42f4a5c9ad07a7: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-d5ecdf2c2cdb4487b0e6315f63e1b1b8 on tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:00:44.274561 19500 ts_tablet_manager.cc:902] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813)
I20240501 08:00:44.275090 19500 tablet_copy_client.cc:308] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:46813
I20240501 08:00:44.275049 19494 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:44.275359 19494 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:00:44.275558 19494 ts_tablet_manager.cc:1367] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.023s user 0.019s sys 0.000s
I20240501 08:00:44.276208 19494 raft_consensus.cc:348] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:44.276335 19494 raft_consensus.cc:729] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:00:44.276440 19494 consensus_queue.cc:260] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:44.276834 19494 ts_tablet_manager.cc:1398] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:44.277432 16583 tablet_copy_service.cc:342] P eca6f08d57d1453784f726199474a671: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-5ae979c5adf44f3e8590ef06c71bf904 received from {username='slave'} at 127.0.99.138:34739
I20240501 08:00:44.277540 16583 tablet_copy_service.cc:433] P eca6f08d57d1453784f726199474a671: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-5ae979c5adf44f3e8590ef06c71bf904 on tablet 5ae979c5adf44f3e8590ef06c71bf904 with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:00:44.279585 17243 tablet_copy_service.cc:140] P 3d06b1f1890443da898fde2eb82daad3: Received BeginTabletCopySession request for tablet 7c25b2699528442f8e2993efbac96bbd from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:44473)
I20240501 08:00:44.279745 17243 tablet_copy_service.cc:161] P 3d06b1f1890443da898fde2eb82daad3: Beginning new tablet copy session on tablet 7c25b2699528442f8e2993efbac96bbd from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:44473: session id = 5aaf071b45ab469cb23321d37fad37f8-7c25b2699528442f8e2993efbac96bbd
I20240501 08:00:44.281129 17243 tablet_copy_source_session.cc:205] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:44.282666 19500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c25b2699528442f8e2993efbac96bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:44.284833 19500 tablet_copy_client.cc:783] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:44.285290 19500 tablet_copy_client.cc:647] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:44.286926 19500 tablet_copy_client.cc:523] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:44.288363 19500 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:00:44.309388 19500 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:44.309728 19500 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:00:44.309947 19500 ts_tablet_manager.cc:1367] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.022s user 0.020s sys 0.000s
I20240501 08:00:44.310444 19500 raft_consensus.cc:348] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:44.310628 19500 raft_consensus.cc:729] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:00:44.310734 19500 consensus_queue.cc:260] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:44.311049 19500 ts_tablet_manager.cc:1398] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:44.312389 17243 tablet_copy_service.cc:342] P 3d06b1f1890443da898fde2eb82daad3: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-7c25b2699528442f8e2993efbac96bbd received from {username='slave'} at 127.0.99.136:44473
I20240501 08:00:44.312513 17243 tablet_copy_service.cc:433] P 3d06b1f1890443da898fde2eb82daad3: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-7c25b2699528442f8e2993efbac96bbd on tablet 7c25b2699528442f8e2993efbac96bbd with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:00:44.446050 18970 raft_consensus.cc:1203] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:44.473927 18839 raft_consensus.cc:1203] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:44.583892 18970 raft_consensus.cc:1203] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:44.587356 19097 raft_consensus.cc:1203] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:44.624714 18970 raft_consensus.cc:1203] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:44.721415 19097 raft_consensus.cc:1203] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:44.744117 19097 raft_consensus.cc:1203] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:44.786477 18839 raft_consensus.cc:1203] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:44.820570 18839 raft_consensus.cc:1203] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:44.921798 19446 raft_consensus.cc:1050] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:00:44.922403 19446 consensus_queue.cc:237] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:44.924760 17089 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:44.925004 16693 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:44.925303 17353 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:44.925683 17220 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:44.925720 16958 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:44.925828 19445 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:44.926220 19445 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:44.926568 18970 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:44.927035 19394 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:44.927464 19394 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:44.927831 19705 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:44.928315 19394 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:44.929200 16827 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:44.930565 19394 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:44.931427 19701 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.932281 16693 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.933701 17089 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.933956 18970 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.936949 16416 catalog_manager.cc:5484] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2 reported cstate change: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.938635 17220 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.938915 17353 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.943006 16958 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.945405 16827 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.947551 16563 consensus_queue.cc:237] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:44.949529 18970 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:44.949529 17220 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:44.949541 16827 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:44.949914 16693 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:44.949995 19704 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:44.950219 19446 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:44.950454 16958 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:44.950819 19446 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:44.951298 19446 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:44.951731 17353 raft_consensus.cc:1261] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:44.952349 19446 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:44.952697 19445 consensus_queue.cc:1035] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:44.955020 19702 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.955446 16693 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.955749 17353 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.957618 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bcce59ab1c5947efb63e2839e30ccc72 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:44.958132 16958 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.958393 16827 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.958829 16413 catalog_manager.cc:5484] T bcce59ab1c5947efb63e2839e30ccc72 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:44.961354 17220 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.963486 18970 raft_consensus.cc:2932] T bcce59ab1c5947efb63e2839e30ccc72 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:44.967188 17071 tablet_service.cc:1514] Processing DeleteTablet for tablet bcce59ab1c5947efb63e2839e30ccc72 with delete_type TABLET_DATA_TOMBSTONED (TS e282199dc2fa4977b83ff07b86cb0690 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:40630
I20240501 08:00:44.968103 19727 tablet_replica.cc:330] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690: stopping tablet replica
I20240501 08:00:44.968585 19727 raft_consensus.cc:2229] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:44.969151 19727 raft_consensus.cc:2258] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:44.970646 19727 ts_tablet_manager.cc:1875] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:44.973210 19727 ts_tablet_manager.cc:1888] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:44.973318 19727 log.cc:1196] T bcce59ab1c5947efb63e2839e30ccc72 P e282199dc2fa4977b83ff07b86cb0690: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/bcce59ab1c5947efb63e2839e30ccc72
I20240501 08:00:44.973910 16408 catalog_manager.cc:4830] TS e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): tablet bcce59ab1c5947efb63e2839e30ccc72 (table rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]) successfully deleted
I20240501 08:00:45.022959 19639 raft_consensus.cc:1050] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:00:45.023633 19639 consensus_queue.cc:237] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:45.025631 16563 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.025801 19703 raft_consensus.cc:1050] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:00:45.025827 16693 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.025885 17353 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.026350 18714 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.026376 19703 consensus_queue.cc:237] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:45.027493 18839 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.027498 19639 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.028482 19639 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.028544 17220 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.028744 16694 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.029223 17090 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.029256 18970 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.029652 19703 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.029700 16827 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.029906 17220 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.030333 19757 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.030407 19703 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.030644 19446 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.031003 16958 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.031522 17089 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.032367 19446 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.032357 19549 raft_consensus.cc:1050] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:00:45.032727 19446 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.033092 19549 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:45.033416 19703 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.033968 17353 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.034085 19639 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.035480 16693 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.035580 19097 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.035710 19446 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.036464 19766 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.037036 16957 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.037351 16827 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.037364 17220 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.038053 16562 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.039024 19754 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.040422 19341 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.040859 19341 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.042006 17091 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.042058 17353 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.042766 19768 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.044224 19754 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.045040 19767 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.045408 19767 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.045691 19767 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.047227 19755 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.048650 19394 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.050192 16563 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.050139 18970 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.051849 16826 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.051908 17221 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.052232 17220 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.053597 16694 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.054278 17354 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.056191 16827 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.056321 16414 catalog_manager.cc:5484] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.056885 16958 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.058246 16420 catalog_manager.cc:5484] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:45.059271 17355 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.059451 16694 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.062073 19767 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.064903 17220 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.064729 19097 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.066665 16957 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.066767 17088 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.069337 17091 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.070844 18839 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.072737 16563 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.075556 16693 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.076221 17353 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.077020 17090 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.080492 19641 raft_consensus.cc:1050] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:00:45.080746 16415 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:45.081182 19641 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:45.083343 16693 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.083791 16957 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.083818 19641 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.083880 17089 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.084288 19385 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.085717 16826 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.085783 16563 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.086000 18970 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.086585 17220 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.084621 19385 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.087160 19641 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.087371 19640 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.087800 19640 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.088204 19640 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.089098 16957 consensus_queue.cc:237] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:45.091483 16693 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.091663 16826 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.092099 16563 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.091711 19640 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.092034 18839 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.092785 16563 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.093705 19639 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.094236 17353 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.095333 16694 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.096628 19639 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.097023 17220 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.099685 17089 raft_consensus.cc:1261] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.100777 19639 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.101133 19757 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.102294 16563 consensus_queue.cc:237] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:45.102209 16416 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:45.102958 19639 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.104887 16824 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.105283 18970 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.105309 17089 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.105655 16693 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.105804 17353 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.105787 19446 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.105979 19703 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.106159 19702 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.106304 19445 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.106936 16957 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.108054 19702 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.108155 18714 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.110180 18969 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.111979 19756 consensus_queue.cc:1035] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.112198 17355 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.114821 16693 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.115207 18839 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.115394 17090 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.116927 16824 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:45.118937 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:45.119165 19703 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.119485 17221 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.119783 17355 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.120122 16420 catalog_manager.cc:5484] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:45.120752 19097 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.121213 19768 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.122267 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet addb23c607754d0aabee4c934f390251 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:45.123433 17201 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e with delete_type TABLET_DATA_TOMBSTONED (TS 3d06b1f1890443da898fde2eb82daad3 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58426
I20240501 08:00:45.123373 17089 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.125097 16823 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.125499 16413 catalog_manager.cc:5484] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:45.127581 17088 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.128046 19768 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.128923 17201 tablet_service.cc:1514] Processing DeleteTablet for tablet addb23c607754d0aabee4c934f390251 with delete_type TABLET_DATA_TOMBSTONED (TS 3d06b1f1890443da898fde2eb82daad3 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58426
I20240501 08:00:45.130801 16826 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.131162 19767 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.131317 19701 raft_consensus.cc:1050] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:00:45.131482 19767 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.131842 19701 consensus_queue.cc:237] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:45.134388 17221 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.134651 19096 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.134766 17354 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.134992 19446 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.135284 16823 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.135390 19446 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.135715 19704 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.135704 17354 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:45.136730 17355 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.137519 16694 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.137845 19702 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.138383 16562 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.138403 18969 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.138830 17087 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.139029 16694 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.139451 19640 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.140079 19445 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.140236 19640 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.141301 16562 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.142812 19640 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.143280 16959 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.143786 19704 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.144826 17091 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.146898 19768 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.148929 19799 tablet_replica.cc:330] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3: stopping tablet replica
I20240501 08:00:45.149361 19799 raft_consensus.cc:2229] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:45.149685 19799 raft_consensus.cc:2258] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:45.150960 16959 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.151654 19702 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.151777 17221 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.152679 16562 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.151670 19766 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.154785 16824 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.154815 16692 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.154810 19790 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.155134 19790 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.156513 16414 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.156811 19640 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.157681 19768 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.157580 16563 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.157770 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6fd34b65ff134bf485af45d1eccd62f9 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:45.159413 16694 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.159639 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fa0d919844a349549b1ad6ffbe440837 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:45.159968 17089 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.160874 16692 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.162081 16419 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:45.163062 19799 ts_tablet_manager.cc:1875] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:45.164732 19097 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.167572 19385 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.168404 18969 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.169004 16563 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.169274 19799 ts_tablet_manager.cc:1888] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:45.169381 19799 log.cc:1196] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 3d06b1f1890443da898fde2eb82daad3: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/c8ae6ca8ef5947f8abe9e92d69d8bb1e
I20240501 08:00:45.169920 19799 tablet_replica.cc:330] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3: stopping tablet replica
I20240501 08:00:45.170004 16407 catalog_manager.cc:4830] TS 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813): tablet c8ae6ca8ef5947f8abe9e92d69d8bb1e (table rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]) successfully deleted
I20240501 08:00:45.170051 19799 raft_consensus.cc:2229] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:45.170226 19799 raft_consensus.cc:2258] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:45.171322 17354 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.172048 19799 ts_tablet_manager.cc:1875] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:45.172596 16959 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.174065 16957 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.174418 17071 tablet_service.cc:1514] Processing DeleteTablet for tablet fa0d919844a349549b1ad6ffbe440837 with delete_type TABLET_DATA_TOMBSTONED (TS e282199dc2fa4977b83ff07b86cb0690 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:40630
I20240501 08:00:45.174896 16939 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd34b65ff134bf485af45d1eccd62f9 with delete_type TABLET_DATA_TOMBSTONED (TS d5a55f6821d84c968a42f4a5c9ad07a7 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33826
I20240501 08:00:45.175143 19394 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.175122 16693 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.175477 19727 tablet_replica.cc:330] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: stopping tablet replica
I20240501 08:00:45.175624 19727 raft_consensus.cc:2229] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:45.175817 19727 raft_consensus.cc:2258] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:45.177233 16824 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.177922 19727 ts_tablet_manager.cc:1875] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:45.178639 16957 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.182031 18970 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.183338 19799 ts_tablet_manager.cc:1888] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:45.183776 19809 tablet_replica.cc:330] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: stopping tablet replica
I20240501 08:00:45.184762 19809 raft_consensus.cc:2229] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:45.184931 19799 log.cc:1196] T addb23c607754d0aabee4c934f390251 P 3d06b1f1890443da898fde2eb82daad3: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/addb23c607754d0aabee4c934f390251
I20240501 08:00:45.184698 19445 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.185343 17091 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.186621 17221 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:45.187397 19727 ts_tablet_manager.cc:1888] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:45.187472 19727 log.cc:1196] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/fa0d919844a349549b1ad6ffbe440837
I20240501 08:00:45.188069 16408 catalog_manager.cc:4830] TS e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): tablet fa0d919844a349549b1ad6ffbe440837 (table rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]) successfully deleted
I20240501 08:00:45.188561 16414 catalog_manager.cc:5484] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.189092 16407 catalog_manager.cc:4830] TS 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813): tablet addb23c607754d0aabee4c934f390251 (table rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]) successfully deleted
I20240501 08:00:45.189393 16695 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.191501 17220 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.189148 19809 raft_consensus.cc:2258] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:45.193259 19809 ts_tablet_manager.cc:1875] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:45.195554 16825 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.195649 19096 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.196979 19809 ts_tablet_manager.cc:1888] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:45.197193 19809 log.cc:1196] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/6fd34b65ff134bf485af45d1eccd62f9
I20240501 08:00:45.198102 16407 catalog_manager.cc:4830] TS d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): tablet 6fd34b65ff134bf485af45d1eccd62f9 (table rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]) successfully deleted
I20240501 08:00:45.198239 17355 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.199369 16563 consensus_queue.cc:237] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:45.200568 16825 raft_consensus.cc:2932] T c8ae6ca8ef5947f8abe9e92d69d8bb1e P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.201853 17223 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.201936 19394 raft_consensus.cc:1050] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:00:45.202085 19096 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.202546 19394 consensus_queue.cc:237] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:45.202797 19705 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.202903 17220 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.203083 19445 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.205569 17089 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.205569 17091 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.206564 16827 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.206895 16695 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.207091 19701 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.207907 16959 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.208245 19096 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.208293 19701 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.208762 16693 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.208925 16825 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.209839 19701 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.210045 19705 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.210537 19702 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.210592 19703 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.210673 19704 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.213219 16957 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.213446 19705 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.215379 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4b8a14c2447e45c5b148ff762bb55c00 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:45.215951 16693 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.216292 16827 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.216337 17221 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.219226 16417 catalog_manager.cc:5484] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 6 to 7, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.220065 17220 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.220652 19705 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.220477 19703 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.221215 17354 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.222090 16957 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.222374 17089 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.222405 16959 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.221853 19703 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.225749 17335 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b8a14c2447e45c5b148ff762bb55c00 with delete_type TABLET_DATA_TOMBSTONED (TS af70005d0c6c48bc9919cae6346f221b not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:56682
I20240501 08:00:45.226327 19821 tablet_replica.cc:330] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b: stopping tablet replica
I20240501 08:00:45.226588 19097 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.227023 19821 raft_consensus.cc:2229] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:45.226941 19445 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.227291 19821 raft_consensus.cc:2258] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:45.229416 17091 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.229650 19705 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.229432 19821 ts_tablet_manager.cc:1875] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:45.230842 16420 catalog_manager.cc:5484] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:45.231423 16959 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.231423 16827 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.234006 19097 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.236083 16695 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.237519 17354 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.239730 17220 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.243002 16563 consensus_queue.cc:237] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:45.245157 16695 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.245503 19097 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.245625 19394 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.245901 16827 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.246402 19394 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.246677 19394 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.247213 17220 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.247327 17091 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.247745 19394 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.248133 17354 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.248759 19394 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.248859 19701 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.250381 19702 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.251493 17091 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.251447 17354 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.253063 16827 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.253635 19097 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.254601 16414 catalog_manager.cc:5484] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.255014 19821 ts_tablet_manager.cc:1888] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:45.255110 19821 log.cc:1196] T 4b8a14c2447e45c5b148ff762bb55c00 P af70005d0c6c48bc9919cae6346f221b: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/4b8a14c2447e45c5b148ff762bb55c00
I20240501 08:00:45.255017 16695 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.255690 16407 catalog_manager.cc:4830] TS af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065): tablet 4b8a14c2447e45c5b148ff762bb55c00 (table rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]) successfully deleted
I20240501 08:00:45.256703 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5ae979c5adf44f3e8590ef06c71bf904 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:45.258833 17220 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:45.260273 16939 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae979c5adf44f3e8590ef06c71bf904 with delete_type TABLET_DATA_TOMBSTONED (TS d5a55f6821d84c968a42f4a5c9ad07a7 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33826
I20240501 08:00:45.260542 19809 tablet_replica.cc:330] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: stopping tablet replica
I20240501 08:00:45.260680 19809 raft_consensus.cc:2229] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:45.260897 19809 raft_consensus.cc:2258] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:45.263429 19809 ts_tablet_manager.cc:1875] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:45.265726 19809 ts_tablet_manager.cc:1888] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:45.265817 19809 log.cc:1196] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/5ae979c5adf44f3e8590ef06c71bf904
I20240501 08:00:45.266273 16827 consensus_queue.cc:237] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:45.266346 16407 catalog_manager.cc:4830] TS d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): tablet 5ae979c5adf44f3e8590ef06c71bf904 (table rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]) successfully deleted
I20240501 08:00:45.268287 17091 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.268329 16959 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.268703 19549 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.268916 16563 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.269035 17220 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.269032 17354 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.269317 19769 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.269465 19768 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.269718 19767 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.269902 19549 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.270395 16695 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.270885 19549 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.273125 19767 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:45.273546 16563 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:45.273787 16693 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:45.276530 17091 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:45.276547 16959 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:45.276880 17355 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:45.278669 16414 catalog_manager.cc:5484] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:45.280408 17220 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:45.285547 16827 consensus_queue.cc:237] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:45.287753 16563 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.288070 16959 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.288071 17091 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.288664 17220 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.288584 19769 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.289150 19525 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.289453 19766 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.289615 19769 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 08:00:45.289780 16761 consensus_peers.cc:485] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab47ecf7f96c4f668750a2bb0bddb1ef. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:45.290510 16693 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.290933 19549 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.292816 17355 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.293349 19549 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.294353 19765 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.294739 17091 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.296849 16405 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ab47ecf7f96c4f668750a2bb0bddb1ef with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:45.296715 16563 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.297106 16693 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.297753 16419 catalog_manager.cc:5484] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.298286 16959 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.300763 17220 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.301679 17355 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.322476 19591 raft_consensus.cc:1050] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:00:45.323086 19591 consensus_queue.cc:237] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:45.325659 17091 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.325687 16827 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.326036 16563 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.326092 19853 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.326650 17355 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.326666 19857 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.326788 16959 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.327265 18839 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.327363 19853 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.327433 16693 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.327517 19857 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.327698 19687 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.328048 19591 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.330622 19856 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.333020 19591 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.333746 17355 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.336285 16563 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.337157 16959 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.337512 16413 catalog_manager.cc:5484] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:45.340168 18839 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.346428 16827 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.347326 17091 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.347470 16693 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.352794 19639 raft_consensus.cc:1050] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:00:45.353320 19639 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:45.355041 18839 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.355144 17220 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.355124 17355 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.355563 19756 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.355648 17221 consensus_queue.cc:237] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:45.355785 19592 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.356580 16827 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.356653 16563 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.357139 18714 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.357676 18714 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.357802 16695 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.358053 17354 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.358546 19639 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.358641 16695 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.358662 18838 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.359046 19870 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.359239 16562 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.359306 17091 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:45.360033 19870 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.360199 19856 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.360304 18714 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.360955 16825 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.361510 19855 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.361668 19870 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.363246 19757 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:45.363312 19858 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.363900 17091 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.364444 19855 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.364910 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7c25b2699528442f8e2993efbac96bbd with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:45.365957 16563 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.366046 19757 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.366683 17091 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.368235 16563 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.369321 16417 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:45.369550 17355 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.369635 17354 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.370615 16413 catalog_manager.cc:5484] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:45.371464 16827 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.372370 17091 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.372637 17221 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.373451 18838 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.373541 16693 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.376575 16695 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.378448 16939 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c25b2699528442f8e2993efbac96bbd with delete_type TABLET_DATA_TOMBSTONED (TS d5a55f6821d84c968a42f4a5c9ad07a7 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33826
I20240501 08:00:45.378701 19809 tablet_replica.cc:330] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7: stopping tablet replica
I20240501 08:00:45.378856 19809 raft_consensus.cc:2229] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:45.379051 19809 raft_consensus.cc:2258] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:45.379824 19809 ts_tablet_manager.cc:1875] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:45.379904 18839 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.380667 16825 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.383746 19809 ts_tablet_manager.cc:1888] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:45.383842 19809 log.cc:1196] T 7c25b2699528442f8e2993efbac96bbd P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/7c25b2699528442f8e2993efbac96bbd
I20240501 08:00:45.386557 16407 catalog_manager.cc:4830] TS d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): tablet 7c25b2699528442f8e2993efbac96bbd (table rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]) successfully deleted
I20240501 08:00:45.386579 16959 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:45.388183 16825 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.388278 16563 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.388492 17354 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.388720 19592 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.389179 19639 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.389727 19639 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.390573 16695 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.390743 18839 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.390970 17221 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:45.391280 19592 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.391469 19754 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.391819 19639 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:45.393292 19639 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.394667 17221 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.395342 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:45.395709 16827 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.395821 16563 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.397715 16414 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.398104 17354 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.398931 18839 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.399030 16695 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:45.403815 17071 tablet_service.cc:1514] Processing DeleteTablet for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with delete_type TABLET_DATA_TOMBSTONED (TS e282199dc2fa4977b83ff07b86cb0690 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:40630
I20240501 08:00:45.404178 19727 tablet_replica.cc:330] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: stopping tablet replica
I20240501 08:00:45.404331 19727 raft_consensus.cc:2229] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:45.404542 19727 raft_consensus.cc:2258] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:45.405524 19727 ts_tablet_manager.cc:1875] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:45.408178 19727 ts_tablet_manager.cc:1888] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:45.408416 19727 log.cc:1196] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/d5ecdf2c2cdb4487b0e6315f63e1b1b8
I20240501 08:00:45.409044 16408 catalog_manager.cc:4830] TS e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 (table rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]) successfully deleted
I20240501 08:00:45.643464 16695 consensus_queue.cc:237] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:45.645649 17354 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.645643 16957 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.645637 17091 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.646064 17220 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.646385 16825 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.646536 19907 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.646917 16563 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.646906 19487 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.647358 19487 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.647902 19907 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.648397 19908 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.650280 19909 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.651306 19487 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.652329 16959 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.653919 17220 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.656986 16416 catalog_manager.cc:5484] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.657554 17091 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.660373 16563 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.662348 17354 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.668713 16825 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.671622 16695 consensus_queue.cc:237] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:45.673617 17354 consensus_queue.cc:237] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:45.674305 16563 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.674774 19689 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.675377 16695 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.675395 16959 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.675655 16957 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.675971 19385 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.676136 16563 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.676146 16827 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.676232 16825 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.676501 17220 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.676898 19385 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.677441 17089 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.677775 17220 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.677762 19640 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.678301 19909 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.678575 19640 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.679229 19909 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.680151 19641 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.680742 19909 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.682318 19641 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.682821 17091 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.683168 17354 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.685284 19916 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.688160 19917 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.690519 16827 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.692293 17089 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
W20240501 08:00:45.694047 16629 consensus_peers.cc:485] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d381bdb28e1f4374b81ab23f6882fe65. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:45.688388 19908 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.695535 16417 catalog_manager.cc:5484] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.698112 16406 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet d381bdb28e1f4374b81ab23f6882fe65 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:45.698407 17221 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.699038 16957 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.699033 16825 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.701784 16695 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.702688 16959 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.704499 19385 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.704840 17221 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.707536 16562 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.707954 16563 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.708623 16416 catalog_manager.cc:5484] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:45.709821 17089 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.712417 17354 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.717115 17354 consensus_queue.cc:237] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:45.720041 17221 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.720320 16957 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.720661 19788 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.720903 19790 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.721694 17089 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.722213 16695 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.722779 19789 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.722817 16825 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.722914 16563 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.723511 19788 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.723843 19788 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.724141 19789 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.725183 19790 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.725533 17221 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.726130 16825 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.728557 16407 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet c6841a66987744f3bc06a4b2bdbc6df2 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:45.729256 16695 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.729416 16957 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.729720 16420 catalog_manager.cc:5484] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.728207 16563 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.736757 17089 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:45.740689 16825 consensus_queue.cc:237] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:45.742769 16695 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.743336 19769 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.743918 16563 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.744246 17089 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.744691 17221 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 08:00:45.744889 17289 consensus_peers.cc:485] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c6841a66987744f3bc06a4b2bdbc6df2. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:45.745021 19769 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.745180 17354 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.745369 19769 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.745868 19549 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.747455 19549 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.748714 16957 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:45.748556 19767 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.749137 19549 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:45.749419 16562 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.751114 17354 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.752844 16957 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.752985 16415 catalog_manager.cc:5484] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:45.754019 16695 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.758201 17089 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.759999 16825 consensus_queue.cc:237] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:45.762442 17091 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.762441 16695 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.762799 17221 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:45.763033 17354 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.763348 19765 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.763492 19341 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 08:00:45.764166 16761 consensus_peers.cc:485] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 99e2e8b8e2024846b0841efc429186ea. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:45.764303 19549 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.765115 17220 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.765676 16562 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.766115 19549 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.767874 19549 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.768345 19341 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.768790 16562 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.769198 16957 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:45.769671 19765 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:45.771143 16405 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 99e2e8b8e2024846b0841efc429186ea with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:45.771548 17354 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.771912 17091 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.772567 16957 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.775204 16416 catalog_manager.cc:5484] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.777238 16695 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.777848 17220 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:45.852730 19975 ts_tablet_manager.cc:902] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:45.853410 19975 tablet_copy_client.cc:308] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:43779
I20240501 08:00:45.854295 16847 tablet_copy_service.cc:140] P 1160c6e72055456992b5d8bf5f3b45c0: Received BeginTabletCopySession request for tablet ab47ecf7f96c4f668750a2bb0bddb1ef from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:45789)
I20240501 08:00:45.854439 16847 tablet_copy_service.cc:161] P 1160c6e72055456992b5d8bf5f3b45c0: Beginning new tablet copy session on tablet ab47ecf7f96c4f668750a2bb0bddb1ef from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:45789: session id = 5f8132e5a8014a6f913ee9ac3338600a-ab47ecf7f96c4f668750a2bb0bddb1ef
I20240501 08:00:45.855619 16847 tablet_copy_source_session.cc:205] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:45.856871 19975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab47ecf7f96c4f668750a2bb0bddb1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:45.859622 19975 tablet_copy_client.cc:783] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:45.859995 19975 tablet_copy_client.cc:647] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:45.861356 19975 tablet_copy_client.cc:523] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:45.862711 19975 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:00:45.883584 19975 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:45.883884 19975 tablet_bootstrap.cc:492] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:00:45.884121 19975 ts_tablet_manager.cc:1367] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.021s user 0.020s sys 0.003s
I20240501 08:00:45.884583 19975 raft_consensus.cc:348] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:45.884728 19975 raft_consensus.cc:729] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:00:45.884846 19975 consensus_queue.cc:260] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:45.885186 19975 ts_tablet_manager.cc:1398] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:45.885926 16847 tablet_copy_service.cc:342] P 1160c6e72055456992b5d8bf5f3b45c0: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-ab47ecf7f96c4f668750a2bb0bddb1ef received from {username='slave'} at 127.0.99.138:45789
I20240501 08:00:45.886045 16847 tablet_copy_service.cc:433] P 1160c6e72055456992b5d8bf5f3b45c0: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-ab47ecf7f96c4f668750a2bb0bddb1ef on tablet ab47ecf7f96c4f668750a2bb0bddb1ef with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:00:46.032486 16695 consensus_queue.cc:237] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:46.034812 17091 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.034812 16957 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.035284 16562 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.035465 16825 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.035542 19907 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.035689 17220 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.035984 19907 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.036087 17354 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.036396 19487 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.036790 19487 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.038879 19487 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.040853 19487 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.042332 19907 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.042690 16562 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.045166 16825 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.045675 16417 catalog_manager.cc:5484] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.046221 17354 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.048153 17220 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.048987 17089 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.051338 16957 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.052186 16693 consensus_queue.cc:237] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
W20240501 08:00:46.054944 16629 consensus_peers.cc:485] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5b060d9fcf294f55b5ccdd3edf5b2886. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:46.056319 16957 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.056432 17354 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.056644 16562 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.056968 17089 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.057155 17220 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.057374 19908 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.057816 19908 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.058168 19908 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.058775 19916 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.058950 16825 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.059108 19909 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.059746 19909 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.062258 19487 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.062737 17089 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.065114 16562 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.065248 17220 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.065529 16406 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:46.065438 16957 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.066394 17354 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.067063 16415 catalog_manager.cc:5484] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:46.070997 16825 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.075835 16825 consensus_queue.cc:237] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:46.077724 17089 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.078238 16957 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.078330 19341 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.078441 16562 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.078617 17220 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.079005 19765 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.079327 19765 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.079900 19341 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.081018 17354 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.081554 19549 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.082497 16695 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.082909 19765 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.083571 19549 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.084044 17354 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.085958 16562 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.087123 17089 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.088587 16415 catalog_manager.cc:5484] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:46.091833 17220 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.095719 16695 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.096158 16957 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:46.097713 16825 consensus_queue.cc:237] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:46.102689 16562 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.102667 16957 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.103061 16695 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.103185 19549 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.103484 19765 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.104311 19341 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.105772 17220 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.106182 19341 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.107645 19341 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.109519 16405 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet b99b55e425f045d7ba4c4133eeae9c23 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
W20240501 08:00:46.110145 16761 consensus_peers.cc:485] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b99b55e425f045d7ba4c4133eeae9c23. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:46.110054 16563 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.110054 16695 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.110628 16957 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.113337 16416 catalog_manager.cc:5484] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:46.112391 17220 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.121404 17089 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.122004 19549 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.122314 17354 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.122778 19549 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.123510 17089 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.128175 17354 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:46.169149 20037 ts_tablet_manager.cc:902] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:46.169766 20037 tablet_copy_client.cc:308] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:39065
I20240501 08:00:46.176512 17375 tablet_copy_service.cc:140] P af70005d0c6c48bc9919cae6346f221b: Received BeginTabletCopySession request for tablet c6841a66987744f3bc06a4b2bdbc6df2 from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:41829)
I20240501 08:00:46.176702 17375 tablet_copy_service.cc:161] P af70005d0c6c48bc9919cae6346f221b: Beginning new tablet copy session on tablet c6841a66987744f3bc06a4b2bdbc6df2 from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:41829: session id = 5aaf071b45ab469cb23321d37fad37f8-c6841a66987744f3bc06a4b2bdbc6df2
I20240501 08:00:46.177716 17375 tablet_copy_source_session.cc:205] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:46.178920 20037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6841a66987744f3bc06a4b2bdbc6df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:46.181953 20037 tablet_copy_client.cc:783] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:46.182186 20037 tablet_copy_client.cc:647] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:46.183576 20037 tablet_copy_client.cc:523] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:46.185027 20037 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:00:46.211074 20037 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:46.211411 20037 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:00:46.211633 20037 ts_tablet_manager.cc:1367] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.027s user 0.027s sys 0.000s
I20240501 08:00:46.212185 20037 raft_consensus.cc:348] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:46.212342 20037 raft_consensus.cc:729] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:00:46.212447 20037 consensus_queue.cc:260] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:46.212867 20037 ts_tablet_manager.cc:1398] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:46.213754 17375 tablet_copy_service.cc:342] P af70005d0c6c48bc9919cae6346f221b: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-c6841a66987744f3bc06a4b2bdbc6df2 received from {username='slave'} at 127.0.99.136:41829
I20240501 08:00:46.213833 17375 tablet_copy_service.cc:433] P af70005d0c6c48bc9919cae6346f221b: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-c6841a66987744f3bc06a4b2bdbc6df2 on tablet c6841a66987744f3bc06a4b2bdbc6df2 with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:00:46.224028 19975 ts_tablet_manager.cc:902] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:46.224543 19975 tablet_copy_client.cc:308] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:43779
I20240501 08:00:46.225047 16847 tablet_copy_service.cc:140] P 1160c6e72055456992b5d8bf5f3b45c0: Received BeginTabletCopySession request for tablet 99e2e8b8e2024846b0841efc429186ea from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:45789)
I20240501 08:00:46.225180 16847 tablet_copy_service.cc:161] P 1160c6e72055456992b5d8bf5f3b45c0: Beginning new tablet copy session on tablet 99e2e8b8e2024846b0841efc429186ea from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:45789: session id = 5f8132e5a8014a6f913ee9ac3338600a-99e2e8b8e2024846b0841efc429186ea
I20240501 08:00:46.226258 16847 tablet_copy_source_session.cc:205] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:46.227646 19975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e2e8b8e2024846b0841efc429186ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:46.230197 19975 tablet_copy_client.cc:783] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:46.230475 19975 tablet_copy_client.cc:647] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:46.231639 19975 tablet_copy_client.cc:523] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:46.237946 19097 raft_consensus.cc:1203] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:46.244364 19975 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:00:46.264650 19975 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:46.264938 19975 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:00:46.265129 19975 ts_tablet_manager.cc:1367] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.002s
I20240501 08:00:46.265614 19975 raft_consensus.cc:348] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:46.265736 19975 raft_consensus.cc:729] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:00:46.265822 19975 consensus_queue.cc:260] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:46.266110 19975 ts_tablet_manager.cc:1398] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:46.266639 16847 tablet_copy_service.cc:342] P 1160c6e72055456992b5d8bf5f3b45c0: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-99e2e8b8e2024846b0841efc429186ea received from {username='slave'} at 127.0.99.138:45789
I20240501 08:00:46.266727 16847 tablet_copy_service.cc:433] P 1160c6e72055456992b5d8bf5f3b45c0: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-99e2e8b8e2024846b0841efc429186ea on tablet 99e2e8b8e2024846b0841efc429186ea with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:00:46.293983 20037 ts_tablet_manager.cc:902] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219)
I20240501 08:00:46.294509 20037 tablet_copy_client.cc:308] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:45219
I20240501 08:00:46.298216 16715 tablet_copy_service.cc:140] P 64196487691e4623b690b2869aad57f7: Received BeginTabletCopySession request for tablet d381bdb28e1f4374b81ab23f6882fe65 from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:57655)
I20240501 08:00:46.298386 16715 tablet_copy_service.cc:161] P 64196487691e4623b690b2869aad57f7: Beginning new tablet copy session on tablet d381bdb28e1f4374b81ab23f6882fe65 from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:57655: session id = 5aaf071b45ab469cb23321d37fad37f8-d381bdb28e1f4374b81ab23f6882fe65
I20240501 08:00:46.299764 16715 tablet_copy_source_session.cc:205] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:46.301148 20037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d381bdb28e1f4374b81ab23f6882fe65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:46.303376 20037 tablet_copy_client.cc:783] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:46.303892 20037 tablet_copy_client.cc:647] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:46.305644 20037 tablet_copy_client.cc:523] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:46.306964 20037 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:00:46.326401 20037 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:46.326720 20037 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:00:46.326895 20037 ts_tablet_manager.cc:1367] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.020s user 0.014s sys 0.004s
I20240501 08:00:46.327322 20037 raft_consensus.cc:348] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:46.327500 20037 raft_consensus.cc:729] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:00:46.327625 20037 consensus_queue.cc:260] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:46.328123 20037 ts_tablet_manager.cc:1398] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 08:00:46.333119 16715 tablet_copy_service.cc:342] P 64196487691e4623b690b2869aad57f7: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-d381bdb28e1f4374b81ab23f6882fe65 received from {username='slave'} at 127.0.99.136:57655
I20240501 08:00:46.333263 16715 tablet_copy_service.cc:433] P 64196487691e4623b690b2869aad57f7: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-d381bdb28e1f4374b81ab23f6882fe65 on tablet d381bdb28e1f4374b81ab23f6882fe65 with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:00:46.634418 19769 raft_consensus.cc:1050] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:00:46.636173 19097 raft_consensus.cc:1203] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:46.639226 19769 consensus_queue.cc:237] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:46.643302 17089 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:46.643636 16957 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:46.643708 19097 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:46.643774 16562 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:46.643834 16695 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:46.644239 20110 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:46.644470 17220 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:46.644512 19549 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:46.644701 19765 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:46.644809 19769 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:46.645052 17354 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:46.645110 20111 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:46.645246 19549 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:46.646661 20110 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:46.648190 18839 raft_consensus.cc:1203] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:46.650630 20107 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.651124 19097 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.652724 16695 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.653015 17354 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.653564 16957 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.654021 16413 catalog_manager.cc:5484] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a reported cstate change: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.655196 16562 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.660398 17220 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.663338 17091 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.671437 20037 ts_tablet_manager.cc:902] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219)
I20240501 08:00:46.671972 20037 tablet_copy_client.cc:308] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:45219
I20240501 08:00:46.672529 16715 tablet_copy_service.cc:140] P 64196487691e4623b690b2869aad57f7: Received BeginTabletCopySession request for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:57655)
I20240501 08:00:46.672642 16715 tablet_copy_service.cc:161] P 64196487691e4623b690b2869aad57f7: Beginning new tablet copy session on tablet 5b060d9fcf294f55b5ccdd3edf5b2886 from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:57655: session id = 5aaf071b45ab469cb23321d37fad37f8-5b060d9fcf294f55b5ccdd3edf5b2886
I20240501 08:00:46.673570 16715 tablet_copy_source_session.cc:205] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:46.675173 20037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b060d9fcf294f55b5ccdd3edf5b2886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:46.676252 16827 consensus_queue.cc:237] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:46.678083 17354 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:46.678231 16957 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:46.678246 17220 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:46.678246 19097 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:46.678845 19769 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:46.679049 20110 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:46.679203 19765 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:46.679432 20107 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:46.682435 20107 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.682881 16695 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:46.683375 19769 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:46.683338 16957 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.684607 16562 raft_consensus.cc:1261] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:46.685047 19549 consensus_queue.cc:1035] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:46.684912 16695 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.685753 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ab47ecf7f96c4f668750a2bb0bddb1ef with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:46.685794 17354 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.686317 16562 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.688565 19097 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.688671 17221 raft_consensus.cc:2932] T ab47ecf7f96c4f668750a2bb0bddb1ef P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:46.689110 16418 catalog_manager.cc:5484] T ab47ecf7f96c4f668750a2bb0bddb1ef P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:46.693380 20037 tablet_copy_client.cc:783] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:46.693647 20037 tablet_copy_client.cc:647] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:46.696415 17071 tablet_service.cc:1514] Processing DeleteTablet for tablet ab47ecf7f96c4f668750a2bb0bddb1ef with delete_type TABLET_DATA_TOMBSTONED (TS e282199dc2fa4977b83ff07b86cb0690 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:40630
I20240501 08:00:46.697964 20037 tablet_copy_client.cc:523] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:46.698303 20138 ts_tablet_manager.cc:902] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:46.699015 20138 tablet_copy_client.cc:308] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:43779
I20240501 08:00:46.700214 20037 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:00:46.703586 20139 tablet_replica.cc:330] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690: stopping tablet replica
I20240501 08:00:46.703606 16847 tablet_copy_service.cc:140] P 1160c6e72055456992b5d8bf5f3b45c0: Received BeginTabletCopySession request for tablet b99b55e425f045d7ba4c4133eeae9c23 from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:35009)
I20240501 08:00:46.703723 16847 tablet_copy_service.cc:161] P 1160c6e72055456992b5d8bf5f3b45c0: Beginning new tablet copy session on tablet b99b55e425f045d7ba4c4133eeae9c23 from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:35009: session id = 5aaf071b45ab469cb23321d37fad37f8-b99b55e425f045d7ba4c4133eeae9c23
I20240501 08:00:46.704002 20139 raft_consensus.cc:2229] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:46.704277 20139 raft_consensus.cc:2258] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:46.704756 16847 tablet_copy_source_session.cc:205] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:46.705772 20138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99b55e425f045d7ba4c4133eeae9c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:46.707427 20139 ts_tablet_manager.cc:1875] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:46.708678 20138 tablet_copy_client.cc:783] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:46.708905 20138 tablet_copy_client.cc:647] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:46.710119 20139 ts_tablet_manager.cc:1888] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:46.710259 20139 log.cc:1196] T ab47ecf7f96c4f668750a2bb0bddb1ef P e282199dc2fa4977b83ff07b86cb0690: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/ab47ecf7f96c4f668750a2bb0bddb1ef
I20240501 08:00:46.711266 16408 catalog_manager.cc:4830] TS e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): tablet ab47ecf7f96c4f668750a2bb0bddb1ef (table rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]) successfully deleted
I20240501 08:00:46.712338 20138 tablet_copy_client.cc:523] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:46.713646 20138 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:00:46.725075 20037 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:46.725471 20037 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:00:46.725666 20037 ts_tablet_manager.cc:1367] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.026s user 0.018s sys 0.003s
I20240501 08:00:46.726125 20037 raft_consensus.cc:348] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:46.726259 20037 raft_consensus.cc:729] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:00:46.726357 20037 consensus_queue.cc:260] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:46.726644 20037 ts_tablet_manager.cc:1398] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:46.727291 16715 tablet_copy_service.cc:342] P 64196487691e4623b690b2869aad57f7: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-5b060d9fcf294f55b5ccdd3edf5b2886 received from {username='slave'} at 127.0.99.136:57655
I20240501 08:00:46.727372 16715 tablet_copy_service.cc:433] P 64196487691e4623b690b2869aad57f7: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-5b060d9fcf294f55b5ccdd3edf5b2886 on tablet 5b060d9fcf294f55b5ccdd3edf5b2886 with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:00:46.729440 18839 raft_consensus.cc:1203] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:46.736203 20138 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:46.736548 20138 tablet_bootstrap.cc:492] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:00:46.736835 20138 ts_tablet_manager.cc:1367] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.023s user 0.022s sys 0.000s
I20240501 08:00:46.737387 20138 raft_consensus.cc:348] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:46.737556 20138 raft_consensus.cc:729] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:00:46.737684 20138 consensus_queue.cc:260] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:46.737963 20138 ts_tablet_manager.cc:1398] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:46.738466 16847 tablet_copy_service.cc:342] P 1160c6e72055456992b5d8bf5f3b45c0: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-b99b55e425f045d7ba4c4133eeae9c23 received from {username='slave'} at 127.0.99.136:35009
I20240501 08:00:46.738579 16847 tablet_copy_service.cc:433] P 1160c6e72055456992b5d8bf5f3b45c0: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-b99b55e425f045d7ba4c4133eeae9c23 on tablet b99b55e425f045d7ba4c4133eeae9c23 with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:00:46.900760 16562 consensus_queue.cc:237] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:46.902592 16827 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.902691 17091 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.902755 17221 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.902930 16957 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.903178 20153 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.903463 19703 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.903584 17354 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.903656 19811 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.903981 19703 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.904074 16695 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:46.904374 19703 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.905275 20153 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:46.906908 19703 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:46.907683 16695 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:46.908749 16957 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:46.910081 17091 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:46.910284 16827 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:46.912561 17354 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:46.913280 16420 catalog_manager.cc:5484] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:46.914242 17221 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:46.919032 16562 consensus_queue.cc:237] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:46.921280 16695 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.921555 17220 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.921625 17091 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.921818 20167 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.922109 20153 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.922554 16957 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.923377 17354 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.923482 16827 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:46.923904 20155 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.924235 19811 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.924444 20155 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:46.924746 20153 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 08:00:46.927211 16496 consensus_peers.cc:485] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b082e9dc1a0144978b1d47337be97b37. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:46.927093 19811 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:46.928129 16827 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:46.928436 16695 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:46.929490 16405 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet b082e9dc1a0144978b1d47337be97b37 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:46.929800 17354 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:46.931053 16417 catalog_manager.cc:5484] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:46.933245 17091 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:46.934329 17220 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:46.936825 16957 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:47.058782 20110 raft_consensus.cc:1050] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:00:47.059291 20110 consensus_queue.cc:237] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:47.061443 16562 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.061503 17354 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.061918 20111 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.062136 16695 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.062170 17091 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.062263 19097 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.062551 20111 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.062726 20109 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.063004 20109 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.063320 20107 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.064195 16957 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.064208 17220 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.065433 20109 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.065645 20111 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.067910 20110 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.068739 17220 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.068776 16957 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.069460 19097 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.071322 17091 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.071439 16562 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.072019 16415 catalog_manager.cc:5484] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.072701 16695 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.075906 17354 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.080785 16827 consensus_queue.cc:237] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:47.082633 16695 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.082615 19097 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.083001 17091 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.083024 17220 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.083266 16562 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.083338 20107 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.083636 17354 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.083765 20107 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.083949 20111 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.084152 19765 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.084347 19769 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.084651 19549 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.086635 19765 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.088059 17220 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.088296 17354 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.088490 17091 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.090092 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 99e2e8b8e2024846b0841efc429186ea with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:47.091111 16562 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.091491 16417 catalog_manager.cc:5484] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.091881 19097 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.093219 16695 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:47.094645 16939 tablet_service.cc:1514] Processing DeleteTablet for tablet 99e2e8b8e2024846b0841efc429186ea with delete_type TABLET_DATA_TOMBSTONED (TS d5a55f6821d84c968a42f4a5c9ad07a7 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33826
I20240501 08:00:47.096442 20203 tablet_replica.cc:330] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7: stopping tablet replica
I20240501 08:00:47.096611 20203 raft_consensus.cc:2229] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:47.096887 20203 raft_consensus.cc:2258] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:47.097680 20203 ts_tablet_manager.cc:1875] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:47.100433 20203 ts_tablet_manager.cc:1888] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:47.100522 20203 log.cc:1196] T 99e2e8b8e2024846b0841efc429186ea P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/99e2e8b8e2024846b0841efc429186ea
I20240501 08:00:47.101114 16407 catalog_manager.cc:4830] TS d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): tablet 99e2e8b8e2024846b0841efc429186ea (table rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]) successfully deleted
I20240501 08:00:47.119243 18839 raft_consensus.cc:1203] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:47.173800 20114 raft_consensus.cc:1050] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:00:47.174460 20114 consensus_queue.cc:237] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:47.176978 16562 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.176978 17091 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.176978 16827 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.177474 18839 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.177435 20210 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.177795 16695 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.177831 20210 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.178615 16957 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.178839 20210 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.179149 20210 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.179437 17220 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.179512 20210 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.180788 20210 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.181219 20211 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.183405 20211 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.184712 17220 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.185768 16562 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.186025 16695 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.186785 16420 catalog_manager.cc:5484] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:47.186806 18839 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.187846 17091 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.197925 16957 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.198619 16827 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.202872 17354 consensus_queue.cc:237] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:47.204883 17220 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.204910 16957 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.204957 16562 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.205308 16827 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.205328 20210 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.205711 16695 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.205991 18839 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.206069 20114 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.207345 20210 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.207600 20213 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.207762 20211 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.209046 20214 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.211082 20213 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.213217 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c6841a66987744f3bc06a4b2bdbc6df2 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:47.213464 17220 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.213495 16825 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.213457 16957 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.214092 18839 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.215534 16562 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.216804 16414 catalog_manager.cc:5484] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 reported cstate change: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.220134 17071 tablet_service.cc:1514] Processing DeleteTablet for tablet c6841a66987744f3bc06a4b2bdbc6df2 with delete_type TABLET_DATA_TOMBSTONED (TS e282199dc2fa4977b83ff07b86cb0690 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:40630
I20240501 08:00:47.221428 20233 tablet_replica.cc:330] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690: stopping tablet replica
I20240501 08:00:47.221828 20233 raft_consensus.cc:2229] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:47.222143 20233 raft_consensus.cc:2258] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:47.223104 16695 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.225648 20233 ts_tablet_manager.cc:1875] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:47.228166 20233 ts_tablet_manager.cc:1888] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:47.228252 20233 log.cc:1196] T c6841a66987744f3bc06a4b2bdbc6df2 P e282199dc2fa4977b83ff07b86cb0690: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/c6841a66987744f3bc06a4b2bdbc6df2
I20240501 08:00:47.228798 16408 catalog_manager.cc:4830] TS e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): tablet c6841a66987744f3bc06a4b2bdbc6df2 (table rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]) successfully deleted
I20240501 08:00:47.251681 18839 raft_consensus.cc:1203] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:47.315724 19916 raft_consensus.cc:1050] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:00:47.316298 19916 consensus_queue.cc:237] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:47.318239 16827 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.318434 17091 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.318531 16957 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.319038 18839 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.319113 16562 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.319129 17221 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.319027 20253 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.319523 20250 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.319769 20251 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.319916 19908 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.320078 17354 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.320108 19909 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.320346 20250 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.320616 20252 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.323881 20252 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.324564 16957 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.324755 16827 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.326617 17221 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.328121 16420 catalog_manager.cc:5484] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:47.328698 17354 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.328795 18839 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.329013 17091 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.332047 16562 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.338299 16693 consensus_queue.cc:237] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:47.340265 16562 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.340445 17221 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.340713 20252 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.340826 18839 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.341166 20252 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.342139 20251 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.342208 17354 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.342558 20252 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.343220 16827 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.343603 20252 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.343842 17091 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.344503 20252 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.345074 20250 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.346002 17354 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.346375 16562 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.346745 16827 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.346813 17091 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.349610 16406 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d381bdb28e1f4374b81ab23f6882fe65 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:47.349925 16417 catalog_manager.cc:5484] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.352067 17221 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.354089 16939 tablet_service.cc:1514] Processing DeleteTablet for tablet d381bdb28e1f4374b81ab23f6882fe65 with delete_type TABLET_DATA_TOMBSTONED (TS d5a55f6821d84c968a42f4a5c9ad07a7 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33826
I20240501 08:00:47.354394 20203 tablet_replica.cc:330] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: stopping tablet replica
I20240501 08:00:47.354550 20203 raft_consensus.cc:2229] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:47.354751 20203 raft_consensus.cc:2258] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:47.355544 20203 ts_tablet_manager.cc:1875] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:47.355720 18839 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.357888 20203 ts_tablet_manager.cc:1888] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:47.358028 20203 log.cc:1196] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/d381bdb28e1f4374b81ab23f6882fe65
I20240501 08:00:47.358592 16407 catalog_manager.cc:4830] TS d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): tablet d381bdb28e1f4374b81ab23f6882fe65 (table rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]) successfully deleted
I20240501 08:00:47.424969 20261 ts_tablet_manager.cc:902] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:00:47.425491 20261 tablet_copy_client.cc:308] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:43741
I20240501 08:00:47.426244 16583 tablet_copy_service.cc:140] P eca6f08d57d1453784f726199474a671: Received BeginTabletCopySession request for tablet b082e9dc1a0144978b1d47337be97b37 from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:51849)
I20240501 08:00:47.426352 16583 tablet_copy_service.cc:161] P eca6f08d57d1453784f726199474a671: Beginning new tablet copy session on tablet b082e9dc1a0144978b1d47337be97b37 from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:51849: session id = 873218458f2346348be2120e543990e2-b082e9dc1a0144978b1d47337be97b37
I20240501 08:00:47.427085 16583 tablet_copy_source_session.cc:205] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:47.428030 20261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b082e9dc1a0144978b1d47337be97b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:47.429986 20261 tablet_copy_client.cc:783] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:47.430151 20261 tablet_copy_client.cc:647] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:47.431356 20261 tablet_copy_client.cc:523] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:47.432593 20261 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:00:47.452665 17221 consensus_queue.cc:237] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } }
I20240501 08:00:47.454964 16827 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:47.454964 16957 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:47.455212 16693 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:47.455211 17091 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:47.455379 16562 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:47.455614 17354 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:47.455963 19858 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:47.456202 20227 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:47.456552 19858 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:47.456848 19858 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:47.457145 19858 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:47.460037 19858 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:47.461725 19858 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:47.462618 16957 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:47.462950 16562 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:47.462957 16827 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:47.464587 16420 catalog_manager.cc:5484] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:47.466106 16693 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:47.466113 17355 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:47.469501 17091 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } }
I20240501 08:00:47.472851 17221 consensus_queue.cc:237] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:47.475081 16693 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:47.475759 19858 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:47.476379 17355 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:47.476537 16957 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:47.477552 20282 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:47.477715 17091 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:47.477909 20282 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:47.478078 20227 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:47.479719 16562 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:47.480192 16827 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:47.480716 20261 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:47.480778 20282 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:47.481034 20261 tablet_bootstrap.cc:492] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:00:47.481230 20261 ts_tablet_manager.cc:1367] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.049s user 0.016s sys 0.005s
I20240501 08:00:47.481679 20261 raft_consensus.cc:348] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:47.481817 20261 raft_consensus.cc:729] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:00:47.481917 20261 consensus_queue.cc:260] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:47.483561 20261 ts_tablet_manager.cc:1398] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240501 08:00:47.483626 16407 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 0d27d6c2cc194773a648909806f4416b with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:47.483865 16693 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:47.480957 19858 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:47.486390 16583 tablet_copy_service.cc:342] P eca6f08d57d1453784f726199474a671: Request end of tablet copy session 873218458f2346348be2120e543990e2-b082e9dc1a0144978b1d47337be97b37 received from {username='slave'} at 127.0.99.137:51849
I20240501 08:00:47.486493 16583 tablet_copy_service.cc:433] P eca6f08d57d1453784f726199474a671: ending tablet copy session 873218458f2346348be2120e543990e2-b082e9dc1a0144978b1d47337be97b37 on tablet b082e9dc1a0144978b1d47337be97b37 with peer 873218458f2346348be2120e543990e2
I20240501 08:00:47.486894 16420 catalog_manager.cc:5484] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:47.487031 16957 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:47.488082 17355 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:47.489708 16827 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:47.491741 17091 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
W20240501 08:00:47.491977 17156 consensus_peers.cc:485] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d27d6c2cc194773a648909806f4416b. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:47.493336 20227 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:47.496488 16562 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:47.517895 20253 raft_consensus.cc:1050] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:00:47.518447 20253 consensus_queue.cc:237] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:47.520880 16562 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.520885 17091 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.520880 16827 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.521065 16959 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.521446 17355 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.521555 20252 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.521605 17221 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.521760 19916 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.521957 20251 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.522111 20250 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.522370 19908 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.523061 19908 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.525182 18839 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.525676 19916 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.528182 20253 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.528972 17355 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.528970 18839 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.530011 16563 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.531028 17091 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.531563 16419 catalog_manager.cc:5484] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 reported cstate change: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.533571 16959 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.534868 16827 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.536774 17221 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.539254 16693 consensus_queue.cc:237] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:47.541368 17355 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.541865 16563 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.541904 19916 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.542215 16957 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.542752 20250 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.543064 20250 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.543648 18839 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.544271 20250 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.545189 16827 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.545578 17221 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.545786 20250 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.546094 20250 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.546419 20250 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.546802 16563 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.547698 16827 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.548368 16406 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:47.549369 16957 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.550122 16414 catalog_manager.cc:5484] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:47.550542 18839 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.553726 17354 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.555078 17071 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 with delete_type TABLET_DATA_TOMBSTONED (TS e282199dc2fa4977b83ff07b86cb0690 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:40630
I20240501 08:00:47.555362 20233 tablet_replica.cc:330] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: stopping tablet replica
I20240501 08:00:47.555497 20233 raft_consensus.cc:2229] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:47.555385 17221 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.555702 20233 raft_consensus.cc:2258] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:47.556820 20233 ts_tablet_manager.cc:1875] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:47.558993 20233 ts_tablet_manager.cc:1888] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:47.559059 20233 log.cc:1196] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/5b060d9fcf294f55b5ccdd3edf5b2886
I20240501 08:00:47.559522 16408 catalog_manager.cc:4830] TS e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): tablet 5b060d9fcf294f55b5ccdd3edf5b2886 (table rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]) successfully deleted
I20240501 08:00:47.749603 17091 consensus_queue.cc:237] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:47.751448 16827 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:47.751578 17354 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:47.752009 20330 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:47.752332 17221 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:47.752390 20332 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:47.752712 16959 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:47.753048 16563 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:47.753405 16693 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:47.753381 20329 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:47.753521 20330 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:47.753772 20332 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:47.753996 20330 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:47.756706 20330 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:47.757477 17221 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:47.757614 17354 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:47.757807 16959 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:47.759821 16563 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:47.760758 16416 catalog_manager.cc:5484] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:47.760825 16693 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:47.763872 16825 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:47.768885 17091 consensus_queue.cc:237] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:47.771085 17354 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:47.771248 16825 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:47.771544 16563 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:47.771708 19803 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:47.771998 17221 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:47.772409 20330 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:47.772557 16959 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:47.773044 16693 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:47.773067 20346 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:47.773458 20346 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:47.774873 20344 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:47.775033 20329 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:47.777995 20344 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
W20240501 08:00:47.778447 17025 consensus_peers.cc:485] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5e0b477de25646eda4f9f0e4921ec118. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:47.778525 17354 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:47.778797 16959 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:47.779400 16825 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:47.780076 16408 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 5e0b477de25646eda4f9f0e4921ec118 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:47.781440 16693 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:47.781725 16417 catalog_manager.cc:5484] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:47.785818 17220 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:47.785972 16563 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:47.803876 18970 raft_consensus.cc:1203] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:47.820547 20107 raft_consensus.cc:1050] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:00:47.821125 20107 consensus_queue.cc:237] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:47.823577 17220 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.823583 16693 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.824055 17091 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.824239 16959 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.824296 16563 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.824410 17354 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.824645 20107 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.824724 18839 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:47.824858 20371 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.825129 19765 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.825309 20370 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.825659 20374 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.825806 20370 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.826756 20372 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:47.829946 20372 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.830510 17354 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.831560 17091 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.832324 18839 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.832430 16563 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.835072 16417 catalog_manager.cc:5484] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.835146 16959 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.838398 16693 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.837285 17220 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.849831 16825 consensus_queue.cc:237] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:47.851874 16563 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.852056 18839 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.852350 20370 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.852561 20371 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.852830 17220 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.853677 20371 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.854029 16693 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.854427 20107 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.854625 17354 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.855232 20371 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.856109 20107 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.856806 16693 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.857620 17220 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.858739 16563 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.858968 18839 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.859756 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b99b55e425f045d7ba4c4133eeae9c23 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:47.861786 16416 catalog_manager.cc:5484] T b99b55e425f045d7ba4c4133eeae9c23 P 5aaf071b45ab469cb23321d37fad37f8 reported cstate change: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.862258 17091 raft_consensus.cc:1261] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:47.862736 20371 consensus_queue.cc:1035] T b99b55e425f045d7ba4c4133eeae9c23 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:47.863821 17354 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.865012 16939 tablet_service.cc:1514] Processing DeleteTablet for tablet b99b55e425f045d7ba4c4133eeae9c23 with delete_type TABLET_DATA_TOMBSTONED (TS d5a55f6821d84c968a42f4a5c9ad07a7 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33826
I20240501 08:00:47.865425 20382 tablet_replica.cc:330] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7: stopping tablet replica
I20240501 08:00:47.866063 20382 raft_consensus.cc:2229] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:47.867087 20382 raft_consensus.cc:2258] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:47.867308 17089 raft_consensus.cc:2932] T b99b55e425f045d7ba4c4133eeae9c23 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:47.869072 20382 ts_tablet_manager.cc:1875] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:47.871011 20382 ts_tablet_manager.cc:1888] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:47.871151 20382 log.cc:1196] T b99b55e425f045d7ba4c4133eeae9c23 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/b99b55e425f045d7ba4c4133eeae9c23
I20240501 08:00:47.871681 16407 catalog_manager.cc:4830] TS d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): tablet b99b55e425f045d7ba4c4133eeae9c23 (table rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]) successfully deleted
I20240501 08:00:47.935271 20261 ts_tablet_manager.cc:902] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813)
I20240501 08:00:47.935736 20261 tablet_copy_client.cc:308] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:46813
I20240501 08:00:47.939379 17243 tablet_copy_service.cc:140] P 3d06b1f1890443da898fde2eb82daad3: Received BeginTabletCopySession request for tablet 0d27d6c2cc194773a648909806f4416b from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:50367)
I20240501 08:00:47.939512 17243 tablet_copy_service.cc:161] P 3d06b1f1890443da898fde2eb82daad3: Beginning new tablet copy session on tablet 0d27d6c2cc194773a648909806f4416b from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:50367: session id = 873218458f2346348be2120e543990e2-0d27d6c2cc194773a648909806f4416b
I20240501 08:00:47.940400 17243 tablet_copy_source_session.cc:205] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:47.941380 20261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d27d6c2cc194773a648909806f4416b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:47.943442 20261 tablet_copy_client.cc:783] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:47.943616 20261 tablet_copy_client.cc:647] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:47.944957 20261 tablet_copy_client.cc:523] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:47.946218 20261 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:00:47.965395 20261 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:47.965677 20261 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:00:47.965849 20261 ts_tablet_manager.cc:1367] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.020s user 0.019s sys 0.000s
I20240501 08:00:47.966325 20261 raft_consensus.cc:348] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:47.966465 20261 raft_consensus.cc:729] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:00:47.966579 20261 consensus_queue.cc:260] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:47.966848 20261 ts_tablet_manager.cc:1398] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:47.967453 17243 tablet_copy_service.cc:342] P 3d06b1f1890443da898fde2eb82daad3: Request end of tablet copy session 873218458f2346348be2120e543990e2-0d27d6c2cc194773a648909806f4416b received from {username='slave'} at 127.0.99.137:50367
I20240501 08:00:47.967535 17243 tablet_copy_service.cc:433] P 3d06b1f1890443da898fde2eb82daad3: ending tablet copy session 873218458f2346348be2120e543990e2-0d27d6c2cc194773a648909806f4416b on tablet 0d27d6c2cc194773a648909806f4416b with peer 873218458f2346348be2120e543990e2
I20240501 08:00:48.219319 20409 ts_tablet_manager.cc:902] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463)
I20240501 08:00:48.219853 20409 tablet_copy_client.cc:308] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:41463
I20240501 08:00:48.223510 17111 tablet_copy_service.cc:140] P e282199dc2fa4977b83ff07b86cb0690: Received BeginTabletCopySession request for tablet 5e0b477de25646eda4f9f0e4921ec118 from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:40869)
I20240501 08:00:48.223670 17111 tablet_copy_service.cc:161] P e282199dc2fa4977b83ff07b86cb0690: Beginning new tablet copy session on tablet 5e0b477de25646eda4f9f0e4921ec118 from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:40869: session id = 5f8132e5a8014a6f913ee9ac3338600a-5e0b477de25646eda4f9f0e4921ec118
I20240501 08:00:48.224939 17111 tablet_copy_source_session.cc:205] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:48.226117 20409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0b477de25646eda4f9f0e4921ec118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:48.228087 20409 tablet_copy_client.cc:783] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:48.228286 20409 tablet_copy_client.cc:647] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:48.229709 20409 tablet_copy_client.cc:523] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:48.231091 20409 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:00:48.252624 20409 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:48.253052 20409 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:00:48.253257 20409 ts_tablet_manager.cc:1367] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.003s
I20240501 08:00:48.253746 20409 raft_consensus.cc:348] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:48.253976 20409 raft_consensus.cc:729] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:00:48.254132 20409 consensus_queue.cc:260] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:48.255442 20409 ts_tablet_manager.cc:1398] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:00:48.256021 17111 tablet_copy_service.cc:342] P e282199dc2fa4977b83ff07b86cb0690: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-5e0b477de25646eda4f9f0e4921ec118 received from {username='slave'} at 127.0.99.138:40869
I20240501 08:00:48.256115 17111 tablet_copy_service.cc:433] P e282199dc2fa4977b83ff07b86cb0690: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-5e0b477de25646eda4f9f0e4921ec118 on tablet 5e0b477de25646eda4f9f0e4921ec118 with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:00:48.258149 17089 consensus_queue.cc:237] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:48.259747 16825 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:48.260015 17221 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:48.260138 20344 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:48.260399 20331 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:48.260452 16959 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:48.260783 17354 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:48.261236 16563 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:48.261592 20329 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:48.261735 20344 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:48.262056 16693 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:48.262068 20331 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:48.262593 20331 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:48.264247 20329 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:48.264789 17221 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:48.265442 16959 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:48.267177 16417 catalog_manager.cc:5484] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } }
I20240501 08:00:48.267727 17354 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:48.268407 16825 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:48.271627 17089 consensus_queue.cc:237] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:48.271986 16563 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:48.274257 16825 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:48.274669 16563 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:48.275121 20346 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:48.275576 16693 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:48.276198 20332 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:48.276253 16959 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:48.276412 17221 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:48.276504 20332 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:48.276774 20344 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:48.276964 17354 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:48.277122 20344 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:48.278697 20346 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:48.278842 16693 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:48.281906 20346 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:48.282492 16957 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:48.282799 16827 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:48.283121 16563 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:48.284965 17221 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:48.285329 16408 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 452b5685e6ca4793a6628d700599bdbb with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:48.284833 16693 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
W20240501 08:00:48.288031 17024 consensus_peers.cc:485] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 452b5685e6ca4793a6628d700599bdbb. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:48.287858 16416 catalog_manager.cc:5484] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:48.292882 17354 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:48.344875 19811 raft_consensus.cc:1050] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:00:48.345451 19811 consensus_queue.cc:237] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:48.347429 18970 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.347556 16959 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.347630 17089 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.347787 16693 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.347805 17221 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.348371 20452 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.348647 16827 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.348647 17354 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.348822 20169 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.348976 20453 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.349150 19811 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.349274 20454 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.349675 20452 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.349929 20169 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.353157 20453 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.353566 17221 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.354508 18970 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.355283 17354 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.356047 16959 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.357029 17089 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.358108 16419 catalog_manager.cc:5484] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.359073 16827 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.361984 16693 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.368569 16563 consensus_queue.cc:237] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:48.370473 16827 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:48.370502 17221 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:48.370504 18970 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:48.371042 19811 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:48.371307 16693 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:48.371322 20456 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:48.371732 16959 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:48.371732 20454 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:48.372117 20454 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:48.372632 20454 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:48.373379 17354 raft_consensus.cc:1261] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:48.374260 20454 consensus_queue.cc:1035] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:48.377120 20451 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.377677 17221 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.377677 18970 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.378793 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b082e9dc1a0144978b1d47337be97b37 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:48.379855 16827 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.381006 16417 catalog_manager.cc:5484] T b082e9dc1a0144978b1d47337be97b37 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.381644 16693 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.382444 17355 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.384425 17071 tablet_service.cc:1514] Processing DeleteTablet for tablet b082e9dc1a0144978b1d47337be97b37 with delete_type TABLET_DATA_TOMBSTONED (TS e282199dc2fa4977b83ff07b86cb0690 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:40630
I20240501 08:00:48.385365 16959 raft_consensus.cc:2932] T b082e9dc1a0144978b1d47337be97b37 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.389209 20463 tablet_replica.cc:330] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690: stopping tablet replica
I20240501 08:00:48.389400 20463 raft_consensus.cc:2229] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:48.389618 20463 raft_consensus.cc:2258] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:48.390290 20463 ts_tablet_manager.cc:1875] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:48.392275 20463 ts_tablet_manager.cc:1888] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:48.392354 20463 log.cc:1196] T b082e9dc1a0144978b1d47337be97b37 P e282199dc2fa4977b83ff07b86cb0690: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/b082e9dc1a0144978b1d47337be97b37
I20240501 08:00:48.392871 16408 catalog_manager.cc:4830] TS e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): tablet b082e9dc1a0144978b1d47337be97b37 (table rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]) successfully deleted
I20240501 08:00:48.452421 18970 raft_consensus.cc:1203] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:48.696288 19097 raft_consensus.cc:1203] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:48.881232 20499 ts_tablet_manager.cc:902] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463)
I20240501 08:00:48.881742 20499 tablet_copy_client.cc:308] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:41463
I20240501 08:00:48.885622 17111 tablet_copy_service.cc:140] P e282199dc2fa4977b83ff07b86cb0690: Received BeginTabletCopySession request for tablet 452b5685e6ca4793a6628d700599bdbb from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:44227)
I20240501 08:00:48.885735 17111 tablet_copy_service.cc:161] P e282199dc2fa4977b83ff07b86cb0690: Beginning new tablet copy session on tablet 452b5685e6ca4793a6628d700599bdbb from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:44227: session id = 873218458f2346348be2120e543990e2-452b5685e6ca4793a6628d700599bdbb
I20240501 08:00:48.886580 17111 tablet_copy_source_session.cc:205] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:48.887499 20499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452b5685e6ca4793a6628d700599bdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:48.888218 20227 raft_consensus.cc:1050] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:00:48.888780 20227 consensus_queue.cc:237] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:48.889964 20499 tablet_copy_client.cc:783] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:48.890128 20499 tablet_copy_client.cc:647] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:48.890801 17354 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.891685 20275 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.892467 17089 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.892764 18970 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.893075 16693 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.893116 20499 tablet_copy_client.cc:523] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:48.893103 16827 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.893539 20274 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.893738 20507 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.894032 16957 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.894074 20507 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.894172 16563 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:48.894567 20499 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:00:48.894604 20506 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.895952 20506 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.896150 20274 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:48.898576 20274 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.900440 16957 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.900760 18970 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.901476 16693 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.902446 16827 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.903545 16417 catalog_manager.cc:5484] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 reported cstate change: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.904578 16563 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.904832 17089 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.907784 17354 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.915367 17221 consensus_queue.cc:237] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:48.917025 17354 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:48.917665 17089 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:48.917732 20227 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:48.918526 16693 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:48.918550 16563 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:48.918859 18970 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:48.918958 20509 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:48.919147 20275 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:48.919405 20508 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:48.919965 20275 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:48.921666 16827 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:48.921813 20507 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.922214 20227 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:48.922322 18970 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.923741 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0d27d6c2cc194773a648909806f4416b with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:48.924010 17089 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.924263 16693 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.925240 16827 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.925944 17354 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.928603 16414 catalog_manager.cc:5484] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.929419 16563 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:48.933980 20499 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:48.934085 16939 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d27d6c2cc194773a648909806f4416b with delete_type TABLET_DATA_TOMBSTONED (TS d5a55f6821d84c968a42f4a5c9ad07a7 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33826
I20240501 08:00:48.934381 20499 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:00:48.934480 20523 tablet_replica.cc:330] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: stopping tablet replica
I20240501 08:00:48.934660 20523 raft_consensus.cc:2229] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:48.934666 20499 ts_tablet_manager.cc:1367] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.040s user 0.028s sys 0.000s
I20240501 08:00:48.934911 20523 raft_consensus.cc:2258] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:48.935366 20499 raft_consensus.cc:348] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:48.935530 20499 raft_consensus.cc:729] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:00:48.935691 20499 consensus_queue.cc:260] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:48.936040 20523 ts_tablet_manager.cc:1875] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:48.937427 20499 ts_tablet_manager.cc:1398] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 08:00:48.938632 17111 tablet_copy_service.cc:342] P e282199dc2fa4977b83ff07b86cb0690: Request end of tablet copy session 873218458f2346348be2120e543990e2-452b5685e6ca4793a6628d700599bdbb received from {username='slave'} at 127.0.99.137:44227
I20240501 08:00:48.938728 17111 tablet_copy_service.cc:433] P e282199dc2fa4977b83ff07b86cb0690: ending tablet copy session 873218458f2346348be2120e543990e2-452b5685e6ca4793a6628d700599bdbb on tablet 452b5685e6ca4793a6628d700599bdbb with peer 873218458f2346348be2120e543990e2
I20240501 08:00:48.939102 20523 ts_tablet_manager.cc:1888] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:48.939190 20523 log.cc:1196] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/0d27d6c2cc194773a648909806f4416b
I20240501 08:00:48.939781 16407 catalog_manager.cc:4830] TS d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): tablet 0d27d6c2cc194773a648909806f4416b (table rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]) successfully deleted
I20240501 08:00:49.189558 16827 consensus_queue.cc:237] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:49.191388 16693 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:49.191795 16563 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:49.191808 16957 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:49.191829 17089 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:49.192193 17354 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:49.192180 20539 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:49.192356 20547 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:49.192576 17221 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:49.192952 20371 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:49.193670 20371 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:49.193972 20371 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:49.194262 20371 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:49.196362 20548 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.196805 16693 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.197352 17089 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.199160 16563 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.199481 16957 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.201409 16415 catalog_manager.cc:5484] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.201710 17221 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.202251 17354 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.207280 16827 consensus_queue.cc:237] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:49.209120 16957 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:49.209673 16693 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:49.210037 17089 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:49.210093 20547 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:49.210376 16563 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:49.210729 20547 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:49.211045 17354 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:49.211817 17221 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:49.212059 20371 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:49.212402 20371 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:49.212697 20371 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:49.212965 20371 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:49.214002 20539 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.215615 17354 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.215615 17221 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.216590 17089 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.217133 16563 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.219347 16413 catalog_manager.cc:5484] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
W20240501 08:00:49.219614 16760 consensus_peers.cc:485] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecd1ff8b89a84669bd8001ea731be7dc. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:49.220314 16693 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.222422 16405 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ecd1ff8b89a84669bd8001ea731be7dc with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:49.226030 16957 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.283429 20332 raft_consensus.cc:1050] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:00:49.284088 20332 consensus_queue.cc:237] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:49.286075 19097 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.286942 16827 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.286906 20581 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.287025 17354 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.287353 16563 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.287421 20581 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.287722 17221 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.287856 20581 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.288252 16957 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.288545 20332 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.288830 20576 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.289072 20577 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.290174 16693 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.290851 20577 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.293043 20332 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.293787 16957 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.293767 16693 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.295145 19097 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.295408 17354 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.296481 16415 catalog_manager.cc:5484] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:49.296656 17221 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.298566 16563 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.298823 16827 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.304425 17089 consensus_queue.cc:237] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:49.306280 17354 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:49.306497 16563 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:49.306497 16827 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:49.307015 20577 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:49.307111 16693 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:49.307369 20581 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:49.307605 20574 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:49.307775 20576 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:49.309542 17221 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:49.309944 20332 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:49.310503 19097 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:49.310447 20581 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.311064 16563 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.312203 16408 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5e0b477de25646eda4f9f0e4921ec118 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:49.312810 16827 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.313657 17354 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.314054 16415 catalog_manager.cc:5484] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:49.316357 20576 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:49.317327 16939 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e0b477de25646eda4f9f0e4921ec118 with delete_type TABLET_DATA_TOMBSTONED (TS d5a55f6821d84c968a42f4a5c9ad07a7 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33826
I20240501 08:00:49.317642 20523 tablet_replica.cc:330] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7: stopping tablet replica
I20240501 08:00:49.317776 20523 raft_consensus.cc:2229] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:49.317950 20523 raft_consensus.cc:2258] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:49.317937 19097 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.320873 17221 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.321383 16693 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:49.325819 20523 ts_tablet_manager.cc:1875] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:49.327734 20523 ts_tablet_manager.cc:1888] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:49.327816 20523 log.cc:1196] T 5e0b477de25646eda4f9f0e4921ec118 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/5e0b477de25646eda4f9f0e4921ec118
I20240501 08:00:49.328440 16407 catalog_manager.cc:4830] TS d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): tablet 5e0b477de25646eda4f9f0e4921ec118 (table rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]) successfully deleted
I20240501 08:00:49.425905 18970 raft_consensus.cc:1203] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:49.606817 20607 ts_tablet_manager.cc:902] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:49.607376 20607 tablet_copy_client.cc:308] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:43779
I20240501 08:00:49.611641 16847 tablet_copy_service.cc:140] P 1160c6e72055456992b5d8bf5f3b45c0: Received BeginTabletCopySession request for tablet ecd1ff8b89a84669bd8001ea731be7dc from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:39189)
I20240501 08:00:49.611784 16847 tablet_copy_service.cc:161] P 1160c6e72055456992b5d8bf5f3b45c0: Beginning new tablet copy session on tablet ecd1ff8b89a84669bd8001ea731be7dc from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:39189: session id = 873218458f2346348be2120e543990e2-ecd1ff8b89a84669bd8001ea731be7dc
I20240501 08:00:49.612702 16847 tablet_copy_source_session.cc:205] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:49.613694 20607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd1ff8b89a84669bd8001ea731be7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:49.615752 20607 tablet_copy_client.cc:783] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:49.615916 20607 tablet_copy_client.cc:647] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:49.617496 20607 tablet_copy_client.cc:523] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:49.619009 20607 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:00:49.640352 20607 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:49.640648 20607 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:00:49.640857 20607 ts_tablet_manager.cc:1367] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.022s user 0.017s sys 0.004s
I20240501 08:00:49.641322 20607 raft_consensus.cc:348] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:49.641445 20607 raft_consensus.cc:729] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:00:49.641543 20607 consensus_queue.cc:260] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:49.641884 20607 ts_tablet_manager.cc:1398] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:00:49.642603 16847 tablet_copy_service.cc:342] P 1160c6e72055456992b5d8bf5f3b45c0: Request end of tablet copy session 873218458f2346348be2120e543990e2-ecd1ff8b89a84669bd8001ea731be7dc received from {username='slave'} at 127.0.99.137:39189
I20240501 08:00:49.642691 16847 tablet_copy_service.cc:433] P 1160c6e72055456992b5d8bf5f3b45c0: ending tablet copy session 873218458f2346348be2120e543990e2-ecd1ff8b89a84669bd8001ea731be7dc on tablet ecd1ff8b89a84669bd8001ea731be7dc with peer 873218458f2346348be2120e543990e2
I20240501 08:00:49.673255 16563 consensus_queue.cc:237] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } }
I20240501 08:00:49.675251 17354 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:49.675253 16695 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:49.675674 16827 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:49.676437 20621 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:49.676687 20622 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:49.676939 20624 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:49.677755 16957 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:49.678330 17089 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:49.678380 17221 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 08:00:49.678750 20621 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:49.678938 20624 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:49.679091 20622 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 08:00:49.681258 20622 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.681761 17089 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.682202 16957 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.683735 17354 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.685317 16419 catalog_manager.cc:5484] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.687602 16693 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.689066 16827 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.694404 16563 consensus_queue.cc:237] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:49.694689 17221 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } }
I20240501 08:00:49.696513 16957 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:49.696656 17354 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:49.697117 20636 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 08:00:49.697889 16496 consensus_peers.cc:485] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8bc2d601fe484082b947235237dbd42f. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:49.698343 16693 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:49.699570 20622 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:49.700743 16827 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:49.702054 20621 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:49.702272 17089 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:49.702430 20621 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:49.702836 20621 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:49.704337 20622 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.706113 16405 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 8bc2d601fe484082b947235237dbd42f with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 08:00:49.707679 17221 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 08:00:49.708297 20453 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 08:00:49.708531 16693 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.708679 17354 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.709609 17089 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.710683 17220 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.712131 16415 catalog_manager.cc:5484] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.716876 16957 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.722492 16827 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:49.801707 20581 raft_consensus.cc:1050] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:00:49.802469 20581 consensus_queue.cc:237] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:49.804807 18970 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.804821 17354 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.804879 17220 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.805248 16957 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.805256 20581 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.805482 20577 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.805675 20332 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.806015 16693 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.806038 20332 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.806288 16563 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.806489 20579 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.806825 20579 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.809739 20579 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:49.810343 17354 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:49.810530 18970 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:49.812397 16693 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:49.812839 17220 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:49.813467 16418 catalog_manager.cc:5484] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 reported cstate change: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:49.813894 16563 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:49.816793 16957 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:49.820463 16825 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:49.821004 20332 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:49.822254 16825 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.102221 18970 raft_consensus.cc:1203] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:50.284737 17089 consensus_queue.cc:237] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:50.286904 18970 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:50.287175 16563 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:50.287389 16693 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:50.287389 20576 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:50.287838 20577 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:50.288125 16825 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:50.288592 17220 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:50.288592 17354 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:50.288995 20581 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:50.289366 20577 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:50.289597 20574 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:50.289841 20332 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:50.292433 20577 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.292888 16563 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.294790 18970 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.294790 17220 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.298137 16413 catalog_manager.cc:5484] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.298367 16693 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.298687 20686 ts_tablet_manager.cc:902] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:00:50.298662 17354 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.299240 20686 tablet_copy_client.cc:308] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:43741
I20240501 08:00:50.300614 16583 tablet_copy_service.cc:140] P eca6f08d57d1453784f726199474a671: Received BeginTabletCopySession request for tablet 8bc2d601fe484082b947235237dbd42f from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:51849)
I20240501 08:00:50.300753 16583 tablet_copy_service.cc:161] P eca6f08d57d1453784f726199474a671: Beginning new tablet copy session on tablet 8bc2d601fe484082b947235237dbd42f from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:51849: session id = 873218458f2346348be2120e543990e2-8bc2d601fe484082b947235237dbd42f
I20240501 08:00:50.301791 16408 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 452b5685e6ca4793a6628d700599bdbb with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:50.301834 16583 tablet_copy_source_session.cc:205] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:50.302318 16939 tablet_service.cc:1514] Processing DeleteTablet for tablet 452b5685e6ca4793a6628d700599bdbb with delete_type TABLET_DATA_TOMBSTONED (TS d5a55f6821d84c968a42f4a5c9ad07a7 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33826
I20240501 08:00:50.305642 20686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bc2d601fe484082b947235237dbd42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:50.305743 16825 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.308894 20686 tablet_copy_client.cc:783] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:50.309128 20686 tablet_copy_client.cc:647] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:50.310523 20686 tablet_copy_client.cc:523] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:50.311298 20691 tablet_replica.cc:330] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7: stopping tablet replica
I20240501 08:00:50.311468 20691 raft_consensus.cc:2229] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:50.311686 20691 raft_consensus.cc:2258] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:50.312752 20691 ts_tablet_manager.cc:1875] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:50.313774 20686 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:00:50.314841 20691 ts_tablet_manager.cc:1888] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:50.314915 20691 log.cc:1196] T 452b5685e6ca4793a6628d700599bdbb P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/452b5685e6ca4793a6628d700599bdbb
I20240501 08:00:50.315436 16407 catalog_manager.cc:4830] TS d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): tablet 452b5685e6ca4793a6628d700599bdbb (table rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]) successfully deleted
I20240501 08:00:50.335842 20686 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:50.336323 20686 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:00:50.336532 20686 ts_tablet_manager.cc:1367] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.023s user 0.022s sys 0.001s
I20240501 08:00:50.337025 20686 raft_consensus.cc:348] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:50.337168 20686 raft_consensus.cc:729] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:00:50.337267 20686 consensus_queue.cc:260] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:50.337653 20686 ts_tablet_manager.cc:1398] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:50.338349 16583 tablet_copy_service.cc:342] P eca6f08d57d1453784f726199474a671: Request end of tablet copy session 873218458f2346348be2120e543990e2-8bc2d601fe484082b947235237dbd42f received from {username='slave'} at 127.0.99.137:51849
I20240501 08:00:50.338434 16583 tablet_copy_service.cc:433] P eca6f08d57d1453784f726199474a671: ending tablet copy session 873218458f2346348be2120e543990e2-8bc2d601fe484082b947235237dbd42f on tablet 8bc2d601fe484082b947235237dbd42f with peer 873218458f2346348be2120e543990e2
I20240501 08:00:50.557917 20560 raft_consensus.cc:1050] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:00:50.558578 20560 consensus_queue.cc:237] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:50.560657 18970 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:50.560758 17089 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:50.561133 20711 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:50.561380 16957 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:50.561465 17221 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:50.561885 20670 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:50.562076 20710 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:50.562242 16693 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:50.562361 20547 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:50.562651 20547 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:50.563061 16563 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:50.563831 20670 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:50.565800 17354 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:50.566267 20711 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:50.568034 20711 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.568512 17089 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.569077 17354 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.570413 16563 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.570917 16693 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.571561 16420 catalog_manager.cc:5484] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.573518 18970 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.574122 16957 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.574558 17221 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.581387 16825 consensus_queue.cc:237] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:50.583248 16563 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:50.583716 17221 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:50.583716 17354 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:50.583894 17089 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:50.583719 20709 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:50.584252 16693 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:50.584224 20710 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:50.584385 20547 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:50.584686 18970 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:50.584733 20560 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:50.584867 20670 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:50.585180 20670 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:50.587422 20711 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.588182 17089 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.588522 16563 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.588522 18970 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.590488 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ecd1ff8b89a84669bd8001ea731be7dc with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:50.590095 16693 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.591038 16413 catalog_manager.cc:5484] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:50.591135 17354 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.592685 17221 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:50.596853 16939 tablet_service.cc:1514] Processing DeleteTablet for tablet ecd1ff8b89a84669bd8001ea731be7dc with delete_type TABLET_DATA_TOMBSTONED (TS d5a55f6821d84c968a42f4a5c9ad07a7 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33826
I20240501 08:00:50.597146 20691 tablet_replica.cc:330] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7: stopping tablet replica
I20240501 08:00:50.597285 20691 raft_consensus.cc:2229] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:50.597458 20691 raft_consensus.cc:2258] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:50.598325 20691 ts_tablet_manager.cc:1875] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:50.601327 20691 ts_tablet_manager.cc:1888] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:50.601411 20691 log.cc:1196] T ecd1ff8b89a84669bd8001ea731be7dc P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/ecd1ff8b89a84669bd8001ea731be7dc
I20240501 08:00:50.601938 16407 catalog_manager.cc:4830] TS d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): tablet ecd1ff8b89a84669bd8001ea731be7dc (table rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]) successfully deleted
I20240501 08:00:50.764465 18970 raft_consensus.cc:1203] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 08:00:51.166851 20624 raft_consensus.cc:1050] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:00:51.167716 20624 consensus_queue.cc:237] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:51.170214 17221 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:51.170214 16693 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:51.170224 16957 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:51.170351 17089 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:51.170666 20742 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:51.170965 20453 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:51.171007 18970 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:51.171314 20453 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:51.171422 16825 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:51.171438 20742 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:51.171679 20744 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:51.173420 17354 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 08:00:51.173560 20741 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:51.174675 20743 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 08:00:51.175118 20453 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.175731 17221 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.176192 17089 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.177142 17354 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.178454 18969 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.178932 16416 catalog_manager.cc:5484] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:51.180629 16825 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.182005 16957 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.183768 16693 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.714792 16563 consensus_queue.cc:237] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:51.716897 16825 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:51.717113 17089 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:51.717540 18969 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:51.718008 20624 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:51.718354 20774 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:51.718664 17354 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:51.718817 17221 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:51.718974 20774 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:51.720738 20774 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:51.720923 20624 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:51.721333 20624 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.721679 16825 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.723124 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8bc2d601fe484082b947235237dbd42f with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:51.725945 16414 catalog_manager.cc:5484] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:51.726047 17354 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.726491 17221 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.727357 18969 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.727495 17089 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.731267 16693 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 08:00:51.731837 20774 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 08:00:51.733480 16693 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:51.735265 16939 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bc2d601fe484082b947235237dbd42f with delete_type TABLET_DATA_TOMBSTONED (TS d5a55f6821d84c968a42f4a5c9ad07a7 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33826
I20240501 08:00:51.735817 20779 tablet_replica.cc:330] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: stopping tablet replica
I20240501 08:00:51.736128 20779 raft_consensus.cc:2229] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:51.736613 20779 raft_consensus.cc:2258] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:51.738473 20779 ts_tablet_manager.cc:1875] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:51.740860 20779 ts_tablet_manager.cc:1888] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 08:00:51.740942 20779 log.cc:1196] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/8bc2d601fe484082b947235237dbd42f
I20240501 08:00:51.742058 16407 catalog_manager.cc:4830] TS d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): tablet 8bc2d601fe484082b947235237dbd42f (table rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]) successfully deleted
I20240501 08:00:52.144163 17201 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:52.149169 19076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:52.152245 17071 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:52.154857 18950 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:52.160521 16807 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:52.160673 16674 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:52.170590 18818 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:52.181118 16541 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:52.191838 17335 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:52.194272 16939 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:52.325984 18950 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:52.330297 16807 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:00:52.334950 17201 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:52.337062 17335 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:52.349309 18818 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:52.352058 19076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:52.353798 16541 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:52.382576 16674 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:52.383805 17071 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:52.387856 16939 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:52.451539 16695 consensus_queue.cc:237] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:52.453934 16563 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.454061 17221 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.454317 18839 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.454438 16825 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.454406 20882 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.454608 16957 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.454916 20882 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.455052 17354 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.455523 19908 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.455842 19908 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.456175 19908 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.456342 20882 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.459260 20882 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.459817 16957 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.460189 16563 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.462657 17354 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.462852 16418 catalog_manager.cc:5484] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.462853 16825 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
W20240501 08:00:52.469255 16629 consensus_peers.cc:485] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5b060d9fcf294f55b5ccdd3edf5b2886. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:52.468964 17221 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.478104 18839 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.487908 17089 consensus_queue.cc:237] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:52.490569 17221 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.491115 16563 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.491679 19097 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.491708 16825 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.492633 16695 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.492653 20574 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.493629 20574 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.494305 20903 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.495393 17354 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.495361 20574 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.495844 20903 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.498802 20903 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.501411 17221 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.500914 20906 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
W20240501 08:00:52.503660 17025 consensus_peers.cc:485] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5e0b477de25646eda4f9f0e4921ec118. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:52.504235 19097 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.504556 16420 catalog_manager.cc:5484] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.506011 16563 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.508877 16695 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.513938 17354 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.514765 16825 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.519131 16563 consensus_queue.cc:237] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
W20240501 08:00:52.521757 16496 consensus_peers.cc:485] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4b8a14c2447e45c5b148ff762bb55c00. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:52.521876 19097 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.521878 16695 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.522104 16825 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.522464 20774 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.522711 20743 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.523087 20773 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.524120 17089 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.524525 20743 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.526499 16957 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.526639 17221 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.526568 20774 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.527071 20773 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.527261 20922 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.528474 16957 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.528450 17221 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.528502 19097 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.531483 16415 catalog_manager.cc:5484] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a reported cstate change: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.531510 17089 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.532276 16825 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.534269 16695 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.547310 16695 consensus_queue.cc:237] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:52.549700 16563 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.549775 16825 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.549908 17089 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.550081 17221 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.550343 18839 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.550513 17354 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.550500 20881 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.550961 20884 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.551134 20885 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 08:00:52.551294 16629 consensus_peers.cc:485] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d381bdb28e1f4374b81ab23f6882fe65. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:52.551482 19908 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.551860 20881 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.553011 20937 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.554970 20937 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.555495 16825 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.556912 18839 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.558559 16413 catalog_manager.cc:5484] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.558542 16563 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.559664 17089 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.564973 17354 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.565546 17221 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.581133 16825 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:52.583182 17354 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.583402 17221 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.584002 16695 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.584196 17089 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.584206 19097 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.584523 20709 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.584619 16563 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 08:00:52.584751 16761 consensus_peers.cc:485] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6fd34b65ff134bf485af45d1eccd62f9. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:52.584766 20953 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.585068 20952 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.585201 20709 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.585425 20711 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.585824 20951 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.587514 20951 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.588141 17089 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.590384 16563 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.590490 17354 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.591801 16416 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.593734 17221 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.596679 19097 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.600524 16695 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.607056 17221 consensus_queue.cc:237] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:52.609553 18839 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.609555 16695 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.610018 17089 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.610023 20969 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.610225 20507 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.610352 17354 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.610438 20967 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 08:00:52.610721 17156 consensus_peers.cc:485] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c25b2699528442f8e2993efbac96bbd. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:52.610929 16563 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.610955 20970 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.611296 20967 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.612592 16825 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.613942 20971 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.613651 20967 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.614068 17089 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.615590 16695 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.616303 16825 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.616777 18839 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.616948 17354 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.618654 16414 catalog_manager.cc:5484] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:52.628278 16562 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:52.635829 17354 consensus_queue.cc:237] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:52.638278 16562 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.638339 16825 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.638717 16695 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.639209 17221 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.639221 16957 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.639854 20214 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.640260 20214 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.641173 18839 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.640604 20214 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20240501 08:00:52.641912 20214 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.642204 20214 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.642482 20214 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 08:00:52.644940 17289 consensus_peers.cc:485] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c6841a66987744f3bc06a4b2bdbc6df2. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:52.646670 20985 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.647184 16562 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.647315 18839 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.647672 17221 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.651047 16420 catalog_manager.cc:5484] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.651124 16695 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.652343 16825 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.658244 16957 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.668905 16562 consensus_queue.cc:237] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:52.671641 17089 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.671722 16695 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.672080 19097 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.672248 20926 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.672677 20926 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.672813 16825 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.673007 17354 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.673238 20743 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.673337 17221 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.673532 20743 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.673820 20743 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.674126 20743 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.677577 20773 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.679327 17354 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.680945 16825 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.681454 16695 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.682555 16415 catalog_manager.cc:5484] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:52.683563 19097 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.686111 17221 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
W20240501 08:00:52.691344 16497 consensus_peers.cc:485] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ae979c5adf44f3e8590ef06c71bf904. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:52.692715 17089 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.699543 17089 consensus_queue.cc:237] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:52.701778 16695 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 08:00:52.702024 17025 consensus_peers.cc:485] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 452b5685e6ca4793a6628d700599bdbb. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:52.702103 16825 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.702180 18969 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.702332 17354 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.702634 20904 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.702872 20914 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.703043 20903 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.703200 20574 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.703305 16562 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.703711 20906 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.704310 17220 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.705427 20906 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.706094 20914 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.706517 16695 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.709499 18969 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.709625 16825 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.709681 16416 catalog_manager.cc:5484] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:52.712421 16562 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.714452 17220 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.717363 17354 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:52.725390 17354 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:52.727814 16695 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.728046 16825 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.729600 18969 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.730482 20984 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.730816 20214 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.731254 20986 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.732069 16957 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.732401 17220 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.732478 20984 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.732859 20984 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 08:00:52.733410 17289 consensus_peers.cc:485] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fa0d919844a349549b1ad6ffbe440837. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:52.733858 16562 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:52.734364 20984 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:52.736904 20985 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.737493 16825 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.738065 16695 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.739288 18969 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.739982 17220 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.740478 16420 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:52.742388 16562 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.742574 16957 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:52.932065 21084 ts_tablet_manager.cc:902] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219)
I20240501 08:00:52.932683 21084 tablet_copy_client.cc:308] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:45219
I20240501 08:00:52.936326 16715 tablet_copy_service.cc:140] P 64196487691e4623b690b2869aad57f7: Received BeginTabletCopySession request for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:47179)
I20240501 08:00:52.936478 16715 tablet_copy_service.cc:161] P 64196487691e4623b690b2869aad57f7: Beginning new tablet copy session on tablet 5b060d9fcf294f55b5ccdd3edf5b2886 from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:47179: session id = 5f8132e5a8014a6f913ee9ac3338600a-5b060d9fcf294f55b5ccdd3edf5b2886
I20240501 08:00:52.937225 16715 tablet_copy_source_session.cc:205] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:52.938143 21084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b060d9fcf294f55b5ccdd3edf5b2886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:52.940164 21084 tablet_copy_client.cc:783] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:52.940487 21084 tablet_copy_client.cc:647] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:52.942106 21084 tablet_copy_client.cc:523] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:52.943591 21084 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:00:52.964427 21084 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:52.964723 21084 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:00:52.964880 21084 ts_tablet_manager.cc:1367] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.021s user 0.018s sys 0.003s
I20240501 08:00:52.965276 21084 raft_consensus.cc:348] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:52.965394 21084 raft_consensus.cc:729] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:00:52.965482 21084 consensus_queue.cc:260] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:52.965742 21084 ts_tablet_manager.cc:1398] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:52.966816 16715 tablet_copy_service.cc:342] P 64196487691e4623b690b2869aad57f7: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-5b060d9fcf294f55b5ccdd3edf5b2886 received from {username='slave'} at 127.0.99.138:47179
I20240501 08:00:52.966931 16715 tablet_copy_service.cc:433] P 64196487691e4623b690b2869aad57f7: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-5b060d9fcf294f55b5ccdd3edf5b2886 on tablet 5b060d9fcf294f55b5ccdd3edf5b2886 with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:00:53.007848 21091 ts_tablet_manager.cc:902] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813)
I20240501 08:00:53.008435 21091 tablet_copy_client.cc:308] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:46813
I20240501 08:00:53.009390 17243 tablet_copy_service.cc:140] P 3d06b1f1890443da898fde2eb82daad3: Received BeginTabletCopySession request for tablet 7c25b2699528442f8e2993efbac96bbd from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:50367)
I20240501 08:00:53.009500 17243 tablet_copy_service.cc:161] P 3d06b1f1890443da898fde2eb82daad3: Beginning new tablet copy session on tablet 7c25b2699528442f8e2993efbac96bbd from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:50367: session id = 873218458f2346348be2120e543990e2-7c25b2699528442f8e2993efbac96bbd
I20240501 08:00:53.010375 17243 tablet_copy_source_session.cc:205] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:53.011469 21091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c25b2699528442f8e2993efbac96bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:53.013756 21091 tablet_copy_client.cc:783] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:53.013921 21091 tablet_copy_client.cc:647] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:53.015067 21091 tablet_copy_client.cc:523] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:53.016252 21091 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:00:53.023826 21084 ts_tablet_manager.cc:902] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219)
I20240501 08:00:53.024343 21084 tablet_copy_client.cc:308] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:45219
I20240501 08:00:53.024858 16715 tablet_copy_service.cc:140] P 64196487691e4623b690b2869aad57f7: Received BeginTabletCopySession request for tablet d381bdb28e1f4374b81ab23f6882fe65 from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:47179)
I20240501 08:00:53.024991 16715 tablet_copy_service.cc:161] P 64196487691e4623b690b2869aad57f7: Beginning new tablet copy session on tablet d381bdb28e1f4374b81ab23f6882fe65 from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:47179: session id = 5f8132e5a8014a6f913ee9ac3338600a-d381bdb28e1f4374b81ab23f6882fe65
I20240501 08:00:53.025748 16715 tablet_copy_source_session.cc:205] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:53.026741 21084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d381bdb28e1f4374b81ab23f6882fe65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:53.029150 21084 tablet_copy_client.cc:783] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:53.029316 21084 tablet_copy_client.cc:647] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:53.030525 21084 tablet_copy_client.cc:523] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:53.032158 21084 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:00:53.034704 21111 ts_tablet_manager.cc:902] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463)
I20240501 08:00:53.035523 21111 tablet_copy_client.cc:308] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:41463
I20240501 08:00:53.043810 17111 tablet_copy_service.cc:140] P e282199dc2fa4977b83ff07b86cb0690: Received BeginTabletCopySession request for tablet 5e0b477de25646eda4f9f0e4921ec118 from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:59535)
I20240501 08:00:53.043979 17111 tablet_copy_service.cc:161] P e282199dc2fa4977b83ff07b86cb0690: Beginning new tablet copy session on tablet 5e0b477de25646eda4f9f0e4921ec118 from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:59535: session id = 5aaf071b45ab469cb23321d37fad37f8-5e0b477de25646eda4f9f0e4921ec118
I20240501 08:00:53.044951 17111 tablet_copy_source_session.cc:205] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:53.046902 21111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0b477de25646eda4f9f0e4921ec118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:53.049388 21111 tablet_copy_client.cc:783] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:53.049575 21111 tablet_copy_client.cc:647] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:53.051015 21111 tablet_copy_client.cc:523] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:53.053225 21111 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:00:53.056905 21091 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:53.057209 21091 tablet_bootstrap.cc:492] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:00:53.057405 21091 ts_tablet_manager.cc:1367] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.041s user 0.024s sys 0.000s
I20240501 08:00:53.057896 21091 raft_consensus.cc:348] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:53.058049 21091 raft_consensus.cc:729] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:00:53.058153 21091 consensus_queue.cc:260] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:53.058449 21091 ts_tablet_manager.cc:1398] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:53.059229 17243 tablet_copy_service.cc:342] P 3d06b1f1890443da898fde2eb82daad3: Request end of tablet copy session 873218458f2346348be2120e543990e2-7c25b2699528442f8e2993efbac96bbd received from {username='slave'} at 127.0.99.137:50367
I20240501 08:00:53.059312 17243 tablet_copy_service.cc:433] P 3d06b1f1890443da898fde2eb82daad3: ending tablet copy session 873218458f2346348be2120e543990e2-7c25b2699528442f8e2993efbac96bbd on tablet 7c25b2699528442f8e2993efbac96bbd with peer 873218458f2346348be2120e543990e2
I20240501 08:00:53.062934 21084 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:53.063347 21084 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:00:53.063530 21084 ts_tablet_manager.cc:1367] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.031s user 0.027s sys 0.000s
I20240501 08:00:53.065480 21084 raft_consensus.cc:348] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:53.065764 21084 raft_consensus.cc:729] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:00:53.065896 21084 consensus_queue.cc:260] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:53.066237 21084 ts_tablet_manager.cc:1398] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 08:00:53.066790 16715 tablet_copy_service.cc:342] P 64196487691e4623b690b2869aad57f7: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-d381bdb28e1f4374b81ab23f6882fe65 received from {username='slave'} at 127.0.99.138:47179
I20240501 08:00:53.066885 16715 tablet_copy_service.cc:433] P 64196487691e4623b690b2869aad57f7: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-d381bdb28e1f4374b81ab23f6882fe65 on tablet d381bdb28e1f4374b81ab23f6882fe65 with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:00:53.082620 21111 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:53.082929 21111 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:00:53.083132 21111 ts_tablet_manager.cc:1367] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.030s user 0.024s sys 0.000s
I20240501 08:00:53.083704 21111 raft_consensus.cc:348] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:53.083856 21111 raft_consensus.cc:729] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:00:53.083986 21111 consensus_queue.cc:260] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:53.084291 21111 ts_tablet_manager.cc:1398] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 08:00:53.085338 17111 tablet_copy_service.cc:342] P e282199dc2fa4977b83ff07b86cb0690: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-5e0b477de25646eda4f9f0e4921ec118 received from {username='slave'} at 127.0.99.136:59535
I20240501 08:00:53.085456 17111 tablet_copy_service.cc:433] P e282199dc2fa4977b83ff07b86cb0690: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-5e0b477de25646eda4f9f0e4921ec118 on tablet 5e0b477de25646eda4f9f0e4921ec118 with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:00:53.113977 21111 ts_tablet_manager.cc:902] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:53.114480 21111 tablet_copy_client.cc:308] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:39065
I20240501 08:00:53.114987 17375 tablet_copy_service.cc:140] P af70005d0c6c48bc9919cae6346f221b: Received BeginTabletCopySession request for tablet fa0d919844a349549b1ad6ffbe440837 from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:41829)
I20240501 08:00:53.115104 17375 tablet_copy_service.cc:161] P af70005d0c6c48bc9919cae6346f221b: Beginning new tablet copy session on tablet fa0d919844a349549b1ad6ffbe440837 from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:41829: session id = 5aaf071b45ab469cb23321d37fad37f8-fa0d919844a349549b1ad6ffbe440837
I20240501 08:00:53.116281 17375 tablet_copy_source_session.cc:205] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:53.117620 21111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0d919844a349549b1ad6ffbe440837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:53.120726 21111 tablet_copy_client.cc:783] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:53.120903 21111 tablet_copy_client.cc:647] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:53.122786 21111 tablet_copy_client.cc:523] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:53.124337 21111 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:00:53.131278 21165 ts_tablet_manager.cc:902] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:00:53.131839 21165 tablet_copy_client.cc:308] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:43741
I20240501 08:00:53.144151 21091 ts_tablet_manager.cc:902] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:00:53.144695 21091 tablet_copy_client.cc:308] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:43741
I20240501 08:00:53.145442 16583 tablet_copy_service.cc:140] P eca6f08d57d1453784f726199474a671: Received BeginTabletCopySession request for tablet 4b8a14c2447e45c5b148ff762bb55c00 from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:51849)
I20240501 08:00:53.145565 16583 tablet_copy_service.cc:161] P eca6f08d57d1453784f726199474a671: Beginning new tablet copy session on tablet 4b8a14c2447e45c5b148ff762bb55c00 from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:51849: session id = 873218458f2346348be2120e543990e2-4b8a14c2447e45c5b148ff762bb55c00
I20240501 08:00:53.146528 16583 tablet_copy_source_session.cc:205] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:53.147600 21091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8a14c2447e45c5b148ff762bb55c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:53.148900 21084 ts_tablet_manager.cc:902] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:00:53.149358 21084 tablet_copy_client.cc:308] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:39065
I20240501 08:00:53.150228 21091 tablet_copy_client.cc:783] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:53.150419 21091 tablet_copy_client.cc:647] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:53.151079 21111 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:53.152650 21091 tablet_copy_client.cc:523] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:53.153851 17375 tablet_copy_service.cc:140] P af70005d0c6c48bc9919cae6346f221b: Received BeginTabletCopySession request for tablet c6841a66987744f3bc06a4b2bdbc6df2 from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:54635)
I20240501 08:00:53.153976 17375 tablet_copy_service.cc:161] P af70005d0c6c48bc9919cae6346f221b: Beginning new tablet copy session on tablet c6841a66987744f3bc06a4b2bdbc6df2 from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:54635: session id = 5f8132e5a8014a6f913ee9ac3338600a-c6841a66987744f3bc06a4b2bdbc6df2
I20240501 08:00:53.154225 21091 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:00:53.155006 17375 tablet_copy_source_session.cc:205] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:53.156049 21084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6841a66987744f3bc06a4b2bdbc6df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:53.156258 21111 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:00:53.156520 21111 ts_tablet_manager.cc:1367] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.032s user 0.026s sys 0.000s
I20240501 08:00:53.157105 21111 raft_consensus.cc:348] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:53.157296 21111 raft_consensus.cc:729] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:00:53.157460 21111 consensus_queue.cc:260] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:53.158329 21084 tablet_copy_client.cc:783] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:53.158488 21084 tablet_copy_client.cc:647] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:53.159256 21111 ts_tablet_manager.cc:1398] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 08:00:53.159718 17375 tablet_copy_service.cc:342] P af70005d0c6c48bc9919cae6346f221b: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-fa0d919844a349549b1ad6ffbe440837 received from {username='slave'} at 127.0.99.136:41829
I20240501 08:00:53.159803 17375 tablet_copy_service.cc:433] P af70005d0c6c48bc9919cae6346f221b: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-fa0d919844a349549b1ad6ffbe440837 on tablet fa0d919844a349549b1ad6ffbe440837 with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:00:53.162420 16583 tablet_copy_service.cc:140] P eca6f08d57d1453784f726199474a671: Received BeginTabletCopySession request for tablet 5ae979c5adf44f3e8590ef06c71bf904 from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:34637)
I20240501 08:00:53.162528 16583 tablet_copy_service.cc:161] P eca6f08d57d1453784f726199474a671: Beginning new tablet copy session on tablet 5ae979c5adf44f3e8590ef06c71bf904 from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:34637: session id = 5aaf071b45ab469cb23321d37fad37f8-5ae979c5adf44f3e8590ef06c71bf904
I20240501 08:00:53.163262 16583 tablet_copy_source_session.cc:205] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:53.164309 21165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae979c5adf44f3e8590ef06c71bf904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:53.165855 21084 tablet_copy_client.cc:523] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:53.166463 21165 tablet_copy_client.cc:783] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:53.166630 21165 tablet_copy_client.cc:647] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:53.167344 21084 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:00:53.167745 21165 tablet_copy_client.cc:523] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:53.168875 21165 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:00:53.185995 21091 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:53.186342 21091 tablet_bootstrap.cc:492] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:00:53.186547 21091 ts_tablet_manager.cc:1367] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.032s user 0.030s sys 0.000s
I20240501 08:00:53.187081 21091 raft_consensus.cc:348] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:53.187237 21091 raft_consensus.cc:729] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:00:53.187343 21091 consensus_queue.cc:260] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:53.187739 21091 ts_tablet_manager.cc:1398] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:00:53.188547 16583 tablet_copy_service.cc:342] P eca6f08d57d1453784f726199474a671: Request end of tablet copy session 873218458f2346348be2120e543990e2-4b8a14c2447e45c5b148ff762bb55c00 received from {username='slave'} at 127.0.99.137:51849
I20240501 08:00:53.188647 16583 tablet_copy_service.cc:433] P eca6f08d57d1453784f726199474a671: ending tablet copy session 873218458f2346348be2120e543990e2-4b8a14c2447e45c5b148ff762bb55c00 on tablet 4b8a14c2447e45c5b148ff762bb55c00 with peer 873218458f2346348be2120e543990e2
I20240501 08:00:53.192399 21165 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:53.192701 21165 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:00:53.192898 21165 ts_tablet_manager.cc:1367] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.024s user 0.023s sys 0.000s
I20240501 08:00:53.193425 21165 raft_consensus.cc:348] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:53.193554 21165 raft_consensus.cc:729] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:00:53.193652 21165 consensus_queue.cc:260] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:53.193933 21165 ts_tablet_manager.cc:1398] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:53.194571 16583 tablet_copy_service.cc:342] P eca6f08d57d1453784f726199474a671: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-5ae979c5adf44f3e8590ef06c71bf904 received from {username='slave'} at 127.0.99.136:34637
I20240501 08:00:53.194654 16583 tablet_copy_service.cc:433] P eca6f08d57d1453784f726199474a671: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-5ae979c5adf44f3e8590ef06c71bf904 on tablet 5ae979c5adf44f3e8590ef06c71bf904 with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:00:53.195484 21084 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:53.196025 21084 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:00:53.196223 21084 ts_tablet_manager.cc:1367] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.029s user 0.021s sys 0.009s
I20240501 08:00:53.196740 21084 raft_consensus.cc:348] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:53.196898 21084 raft_consensus.cc:729] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:00:53.197017 21084 consensus_queue.cc:260] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:53.197350 21084 ts_tablet_manager.cc:1398] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:53.197881 17375 tablet_copy_service.cc:342] P af70005d0c6c48bc9919cae6346f221b: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-c6841a66987744f3bc06a4b2bdbc6df2 received from {username='slave'} at 127.0.99.138:54635
I20240501 08:00:53.197953 17375 tablet_copy_service.cc:433] P af70005d0c6c48bc9919cae6346f221b: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-c6841a66987744f3bc06a4b2bdbc6df2 on tablet c6841a66987744f3bc06a4b2bdbc6df2 with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:00:53.200800 21165 ts_tablet_manager.cc:902] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:53.201318 21165 tablet_copy_client.cc:308] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:43779
I20240501 08:00:53.201925 16847 tablet_copy_service.cc:140] P 1160c6e72055456992b5d8bf5f3b45c0: Received BeginTabletCopySession request for tablet 6fd34b65ff134bf485af45d1eccd62f9 from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:35009)
I20240501 08:00:53.202062 16847 tablet_copy_service.cc:161] P 1160c6e72055456992b5d8bf5f3b45c0: Beginning new tablet copy session on tablet 6fd34b65ff134bf485af45d1eccd62f9 from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:35009: session id = 5aaf071b45ab469cb23321d37fad37f8-6fd34b65ff134bf485af45d1eccd62f9
I20240501 08:00:53.202656 21084 ts_tablet_manager.cc:902] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463)
I20240501 08:00:53.202999 16847 tablet_copy_source_session.cc:205] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:53.203125 21084 tablet_copy_client.cc:308] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:41463
I20240501 08:00:53.203639 17111 tablet_copy_service.cc:140] P e282199dc2fa4977b83ff07b86cb0690: Received BeginTabletCopySession request for tablet 452b5685e6ca4793a6628d700599bdbb from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:40869)
I20240501 08:00:53.203752 17111 tablet_copy_service.cc:161] P e282199dc2fa4977b83ff07b86cb0690: Beginning new tablet copy session on tablet 452b5685e6ca4793a6628d700599bdbb from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:40869: session id = 5f8132e5a8014a6f913ee9ac3338600a-452b5685e6ca4793a6628d700599bdbb
I20240501 08:00:53.204092 21165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd34b65ff134bf485af45d1eccd62f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:53.204743 17111 tablet_copy_source_session.cc:205] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:53.205824 21084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452b5685e6ca4793a6628d700599bdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:53.206337 21165 tablet_copy_client.cc:783] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:53.206496 21165 tablet_copy_client.cc:647] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:53.207782 21084 tablet_copy_client.cc:783] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:53.207964 21084 tablet_copy_client.cc:647] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:53.209475 21084 tablet_copy_client.cc:523] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:53.209517 21165 tablet_copy_client.cc:523] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:53.211021 21084 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:00:53.211021 21165 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:00:53.241215 21084 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:53.241359 21165 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:53.241590 21165 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:00:53.241618 21084 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:00:53.241787 21084 ts_tablet_manager.cc:1367] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.031s user 0.027s sys 0.002s
I20240501 08:00:53.241787 21165 ts_tablet_manager.cc:1367] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.031s user 0.029s sys 0.001s
I20240501 08:00:53.242291 21165 raft_consensus.cc:348] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:53.242291 21084 raft_consensus.cc:348] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:53.242445 21165 raft_consensus.cc:729] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:00:53.242445 21084 raft_consensus.cc:729] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:00:53.242550 21165 consensus_queue.cc:260] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:53.242561 21084 consensus_queue.cc:260] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:53.242864 21165 ts_tablet_manager.cc:1398] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:53.242982 21084 ts_tablet_manager.cc:1398] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 08:00:53.243494 16847 tablet_copy_service.cc:342] P 1160c6e72055456992b5d8bf5f3b45c0: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-6fd34b65ff134bf485af45d1eccd62f9 received from {username='slave'} at 127.0.99.136:35009
I20240501 08:00:53.243579 16847 tablet_copy_service.cc:433] P 1160c6e72055456992b5d8bf5f3b45c0: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-6fd34b65ff134bf485af45d1eccd62f9 on tablet 6fd34b65ff134bf485af45d1eccd62f9 with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:00:53.244342 17111 tablet_copy_service.cc:342] P e282199dc2fa4977b83ff07b86cb0690: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-452b5685e6ca4793a6628d700599bdbb received from {username='slave'} at 127.0.99.138:40869
I20240501 08:00:53.244429 17111 tablet_copy_service.cc:433] P e282199dc2fa4977b83ff07b86cb0690: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-452b5685e6ca4793a6628d700599bdbb on tablet 452b5685e6ca4793a6628d700599bdbb with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:00:53.405539 19097 raft_consensus.cc:1203] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:53.500919 18969 raft_consensus.cc:1203] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:53.512214 19097 raft_consensus.cc:1203] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:53.532127 19097 raft_consensus.cc:1203] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:53.560724 18839 raft_consensus.cc:1203] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:53.622530 18839 raft_consensus.cc:1203] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:53.702790 18839 raft_consensus.cc:1203] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:53.742549 19097 raft_consensus.cc:1203] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:53.752192 18970 raft_consensus.cc:1203] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:53.818466 18839 raft_consensus.cc:1203] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:53.908489 21123 raft_consensus.cc:1050] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:00:53.909142 21123 consensus_queue.cc:237] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:53.911727 16825 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:53.912001 16562 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:53.912526 17355 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:53.912705 17220 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:53.913085 20884 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:53.913254 17089 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:53.913342 21313 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:53.913826 21313 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:53.914180 21313 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:53.914381 20884 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:53.914508 19097 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:53.914984 18839 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:53.916306 21310 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:53.918129 20884 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:53.918659 20884 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.919298 16562 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.921136 17355 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.922456 16827 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.922760 16416 catalog_manager.cc:5484] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:53.923697 18839 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.924453 17089 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.926569 19097 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.929041 17220 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.935619 16695 consensus_queue.cc:237] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:53.937551 16562 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:53.937992 19097 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:53.938181 21313 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:53.940090 17355 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:53.940567 17089 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:53.940747 21313 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:53.940994 21123 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:53.941470 18839 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:53.941967 21123 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:53.942306 17220 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:53.942888 20884 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:53.943248 21313 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:53.944937 21310 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.945319 17089 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.946270 17220 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.948400 16420 catalog_manager.cc:5484] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.948557 18839 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.948748 16562 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.951470 16406 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d381bdb28e1f4374b81ab23f6882fe65 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:53.951809 19097 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.952014 17355 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:53.952893 16807 tablet_service.cc:1514] Processing DeleteTablet for tablet d381bdb28e1f4374b81ab23f6882fe65 with delete_type TABLET_DATA_TOMBSTONED (TS 1160c6e72055456992b5d8bf5f3b45c0 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:39058
I20240501 08:00:53.956178 21337 tablet_replica.cc:330] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0: stopping tablet replica
I20240501 08:00:53.956561 21337 raft_consensus.cc:2229] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:53.958300 21337 raft_consensus.cc:2258] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:53.960897 21337 ts_tablet_manager.cc:1875] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:53.964998 21337 ts_tablet_manager.cc:1888] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 08:00:53.965139 21337 log.cc:1196] T d381bdb28e1f4374b81ab23f6882fe65 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/d381bdb28e1f4374b81ab23f6882fe65
I20240501 08:00:53.965803 16405 catalog_manager.cc:4830] TS 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): tablet d381bdb28e1f4374b81ab23f6882fe65 (table rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]) successfully deleted
I20240501 08:00:54.011838 16957 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:54.014031 17355 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.014106 17220 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.014863 18839 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.014983 21376 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.015125 16693 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.015328 21242 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.015630 21242 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.015901 16562 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.015895 16827 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.016752 20810 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.016934 21376 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.017124 21375 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.020169 21377 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.021224 16693 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.023056 16562 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
W20240501 08:00:54.024251 16893 consensus_peers.cc:485] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5ecdf2c2cdb4487b0e6315f63e1b1b8. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:54.024852 16420 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:54.028426 17355 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.031212 21241 raft_consensus.cc:1050] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:00:54.031692 21298 raft_consensus.cc:1050] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:00:54.031811 21241 consensus_queue.cc:237] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:54.032392 21298 consensus_queue.cc:237] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:54.032637 18839 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.034533 19097 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.034688 19096 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.035044 21298 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.035239 17354 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.035288 16693 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.035494 16562 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.036237 17220 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.036795 16562 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.036948 17354 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.037240 18838 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.037286 16827 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.037881 16957 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.037838 20884 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.038233 20884 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.038486 20884 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.038722 16825 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.040418 21393 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.040874 21393 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.041172 21393 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.041404 21245 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.042484 21123 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.042894 21123 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.043684 17220 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.043956 17223 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.046206 16827 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.048216 18839 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.048157 21395 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.048707 20884 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.051226 21393 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.053027 21391 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.054437 21394 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.055464 20969 raft_consensus.cc:1050] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:00:54.055423 19096 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.056093 20969 consensus_queue.cc:237] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:54.058558 16413 catalog_manager.cc:5484] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:54.058809 18838 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.059770 17353 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.060827 18837 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.062319 17355 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.063818 21417 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.063810 21123 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.064302 21417 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.064692 17221 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.065016 18970 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.065747 16824 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.067562 19096 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.068830 18839 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.069062 17089 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.070547 16562 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.071507 16827 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.076004 21416 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.067840 16417 catalog_manager.cc:5484] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:54.076622 21415 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.078645 17089 consensus_queue.cc:237] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:54.078543 16562 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.078954 16693 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.079917 16563 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.080181 21415 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.080732 21415 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.082882 17355 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.083264 19096 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.083439 16957 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.084619 18839 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.084666 21395 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.085093 21395 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.086621 16695 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.086961 16693 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.089165 17223 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.089630 16827 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.089615 21393 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.089736 21415 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.089941 20743 raft_consensus.cc:1050] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:00:54.090101 21394 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.090464 20743 consensus_queue.cc:237] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:54.092900 17354 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.093271 17089 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.093281 17220 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.093686 21251 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.094214 21251 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.095232 16562 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.095623 17353 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.095993 19097 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.095831 18970 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.096105 21396 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.096674 20926 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.097585 16695 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.099066 21251 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.099217 17354 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.099257 16419 catalog_manager.cc:5484] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.100045 16695 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.100777 18838 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.100903 21251 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.101621 16824 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.102161 21251 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.102833 16825 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.104660 17091 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.108419 20969 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.108556 17221 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.110090 21251 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.110563 21396 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.113087 16827 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.114689 16692 consensus_queue.cc:237] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:54.114972 21394 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.115460 17223 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.117795 16408 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5e0b477de25646eda4f9f0e4921ec118 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:54.118204 17354 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.118297 17223 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.118701 21244 raft_consensus.cc:1050] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:00:54.118701 18834 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.119259 20884 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.119258 21244 consensus_queue.cc:237] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:54.119724 20884 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.121327 19097 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.121836 20884 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.122779 16562 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.122836 19102 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.122493 16420 catalog_manager.cc:5484] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.123284 21243 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.124696 20984 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.125902 16959 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.126003 16541 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e0b477de25646eda4f9f0e4921ec118 with delete_type TABLET_DATA_TOMBSTONED (TS eca6f08d57d1453784f726199474a671 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:50112
I20240501 08:00:54.126542 16957 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.126986 18834 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.127060 21243 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.127398 18839 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.128324 21450 tablet_replica.cc:330] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: stopping tablet replica
I20240501 08:00:54.128657 21450 raft_consensus.cc:2229] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:54.128854 21310 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.128970 21450 raft_consensus.cc:2258] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:54.129257 21310 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.130445 21450 ts_tablet_manager.cc:1875] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:54.130942 21396 raft_consensus.cc:1050] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:00:54.131172 21243 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.131542 21396 consensus_queue.cc:237] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:54.132103 16692 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.132513 21243 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.134781 16826 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.135054 17221 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.135324 21393 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.135540 20984 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.135604 20884 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.135917 16562 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.136267 18970 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.136359 21310 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.136878 16824 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.137290 20984 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.138095 16563 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.138648 16562 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.138898 17222 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.139016 21245 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.139267 21395 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.139993 17354 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.140588 19096 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.141423 17353 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.142294 21395 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.143046 20905 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.144804 16695 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.145244 21393 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.146876 16827 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.148481 19096 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.149492 21393 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.151154 18837 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.152740 19097 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.154619 16693 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.155721 16959 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.158232 17223 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.158778 16413 catalog_manager.cc:5484] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a reported cstate change: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.159373 16406 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:54.160601 17354 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.161365 18837 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.163337 16807 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 with delete_type TABLET_DATA_TOMBSTONED (TS 1160c6e72055456992b5d8bf5f3b45c0 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:39058
I20240501 08:00:54.163576 21337 tablet_replica.cc:330] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0: stopping tablet replica
I20240501 08:00:54.163729 21337 raft_consensus.cc:2229] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:54.164049 21337 raft_consensus.cc:2258] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:54.165319 21337 ts_tablet_manager.cc:1875] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:54.165194 21427 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.167640 16825 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.169370 21337 ts_tablet_manager.cc:1888] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 08:00:54.169513 21337 log.cc:1196] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/5b060d9fcf294f55b5ccdd3edf5b2886
I20240501 08:00:54.170140 16405 catalog_manager.cc:4830] TS 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): tablet 5b060d9fcf294f55b5ccdd3edf5b2886 (table rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]) successfully deleted
I20240501 08:00:54.170760 16413 catalog_manager.cc:5484] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.172300 18838 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.172394 21444 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.175316 16562 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.175400 16825 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.177743 16418 catalog_manager.cc:5484] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:54.178651 19097 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.179724 17354 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.182353 17221 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.183207 16692 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.183475 16693 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.186589 19097 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.188894 17220 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.193078 21456 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.193842 17222 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.193842 16563 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.194422 21450 ts_tablet_manager.cc:1888] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 08:00:54.194540 21450 log.cc:1196] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/5e0b477de25646eda4f9f0e4921ec118
I20240501 08:00:54.195837 16405 catalog_manager.cc:4830] TS eca6f08d57d1453784f726199474a671 (127.0.99.129:43741): tablet 5e0b477de25646eda4f9f0e4921ec118 (table rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]) successfully deleted
I20240501 08:00:54.197504 16413 catalog_manager.cc:5484] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.197646 16695 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.199036 18839 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.201660 18970 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.202414 17354 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.204277 16825 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.207370 19102 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.211371 17089 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.211789 16957 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.219380 21444 raft_consensus.cc:1050] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:00:54.221138 21444 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:54.221683 17354 consensus_queue.cc:237] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:54.224218 16957 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.224350 17222 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.225221 21441 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.225355 16563 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.225445 16695 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.225639 21441 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.225950 19102 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.226312 18839 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.226677 18970 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.226924 16825 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.227442 21440 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.227733 16695 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.227803 21440 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.228149 21440 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.228452 18839 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.228958 21442 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.229213 20984 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.229295 21442 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.229413 16562 consensus_queue.cc:237] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:54.229808 16563 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.231055 21244 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.231177 17089 consensus_queue.cc:237] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:54.231426 21244 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.231585 17353 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.231623 17220 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.231734 21244 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.231981 20743 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.232170 20926 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.233947 17091 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.234123 19102 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.234663 20743 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.234798 18969 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.235334 20905 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.235702 20905 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.236514 16826 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.237533 21442 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.238849 21245 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.238775 17220 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.241914 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c6841a66987744f3bc06a4b2bdbc6df2 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:54.242414 18838 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.242780 19102 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.242725 16413 catalog_manager.cc:5484] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.243396 16692 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.243436 16693 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.244042 18838 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.244228 17353 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.244391 16694 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.245175 21454 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.248621 21299 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.248854 16957 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.249994 21454 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.252779 17222 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.254242 21443 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.254369 17089 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.255064 20905 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.255195 19102 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.256008 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5ae979c5adf44f3e8590ef06c71bf904 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:54.256718 17221 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.258386 18969 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.259514 16807 tablet_service.cc:1514] Processing DeleteTablet for tablet c6841a66987744f3bc06a4b2bdbc6df2 with delete_type TABLET_DATA_TOMBSTONED (TS 1160c6e72055456992b5d8bf5f3b45c0 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:39058
I20240501 08:00:54.259897 21337 tablet_replica.cc:330] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0: stopping tablet replica
I20240501 08:00:54.260093 21337 raft_consensus.cc:2229] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:54.260371 21337 raft_consensus.cc:2258] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:54.260274 16826 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.261066 16413 catalog_manager.cc:5484] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:54.261236 16420 catalog_manager.cc:5484] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 reported cstate change: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.261868 16408 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 452b5685e6ca4793a6628d700599bdbb with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:54.263031 21337 ts_tablet_manager.cc:1875] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:54.262965 17353 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.265770 21337 ts_tablet_manager.cc:1888] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 08:00:54.265851 21337 log.cc:1196] T c6841a66987744f3bc06a4b2bdbc6df2 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/c6841a66987744f3bc06a4b2bdbc6df2
I20240501 08:00:54.266386 16405 catalog_manager.cc:4830] TS 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): tablet c6841a66987744f3bc06a4b2bdbc6df2 (table rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]) successfully deleted
I20240501 08:00:54.266271 16693 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.266613 20926 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.266839 20743 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.267093 20926 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.267740 17220 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.268302 16957 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.268746 21394 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.269503 21443 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.270246 16807 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae979c5adf44f3e8590ef06c71bf904 with delete_type TABLET_DATA_TOMBSTONED (TS 1160c6e72055456992b5d8bf5f3b45c0 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:39058
I20240501 08:00:54.271945 16562 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.273223 17220 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.275389 16541 tablet_service.cc:1514] Processing DeleteTablet for tablet 452b5685e6ca4793a6628d700599bdbb with delete_type TABLET_DATA_TOMBSTONED (TS eca6f08d57d1453784f726199474a671 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:50112
I20240501 08:00:54.278807 19102 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.280746 18838 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.280747 17354 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.281229 16693 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.283403 21450 tablet_replica.cc:330] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: stopping tablet replica
I20240501 08:00:54.283649 21450 raft_consensus.cc:2229] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:54.284126 21450 raft_consensus.cc:2258] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:54.284860 21337 tablet_replica.cc:330] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0: stopping tablet replica
I20240501 08:00:54.285029 21337 raft_consensus.cc:2229] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:54.285305 21337 raft_consensus.cc:2258] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:54.286188 21337 ts_tablet_manager.cc:1875] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:54.291824 21337 ts_tablet_manager.cc:1888] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 08:00:54.292047 21337 log.cc:1196] T 5ae979c5adf44f3e8590ef06c71bf904 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/5ae979c5adf44f3e8590ef06c71bf904
I20240501 08:00:54.292330 21450 ts_tablet_manager.cc:1875] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:54.292979 16405 catalog_manager.cc:4830] TS 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): tablet 5ae979c5adf44f3e8590ef06c71bf904 (table rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]) successfully deleted
I20240501 08:00:54.293076 19102 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:54.296274 21450 ts_tablet_manager.cc:1888] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 08:00:54.300690 21244 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.303308 17221 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.303543 16957 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.304723 16825 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.305614 18969 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.300360 21429 raft_consensus.cc:1050] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:00:54.309695 21429 consensus_queue.cc:237] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:54.310592 16695 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.310588 18839 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.305486 16563 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.313171 16413 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.314496 18969 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.316216 21430 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.317600 17221 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.317802 17089 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.318397 21251 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.318882 21251 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.321099 16825 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.321223 16957 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.322228 20926 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.322620 20926 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.323915 21300 raft_consensus.cc:1050] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:00:54.324597 21300 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:54.327709 21450 log.cc:1196] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/452b5685e6ca4793a6628d700599bdbb
I20240501 08:00:54.328397 16695 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.328476 16405 catalog_manager.cc:4830] TS eca6f08d57d1453784f726199474a671 (127.0.99.129:43741): tablet 452b5685e6ca4793a6628d700599bdbb (table rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]) successfully deleted
I20240501 08:00:54.329715 17354 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.330426 17221 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.331072 16562 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.331182 20743 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.331389 17354 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:54.331987 20953 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.332674 19097 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.334820 16959 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.334887 18839 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.335238 21444 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.335698 21243 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.336223 18970 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.336836 21428 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.337136 16562 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.337007 17220 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.337508 21243 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.338001 21252 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.338397 21252 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.339057 17089 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.339666 16695 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.340076 17221 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.340824 17089 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.343308 16957 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.343472 18969 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.344301 21444 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.345026 21444 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.345610 18839 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.346629 16418 catalog_manager.cc:5484] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.347579 20953 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.348086 20953 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.348393 20953 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.349449 20984 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.350340 16695 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.353731 19102 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.355427 21430 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.357986 20984 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.361604 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fa0d919844a349549b1ad6ffbe440837 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:54.362473 16959 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.362669 19102 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.366899 16693 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.369479 16417 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:54.373209 16825 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.373591 16807 tablet_service.cc:1514] Processing DeleteTablet for tablet fa0d919844a349549b1ad6ffbe440837 with delete_type TABLET_DATA_TOMBSTONED (TS 1160c6e72055456992b5d8bf5f3b45c0 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:39058
I20240501 08:00:54.373812 21337 tablet_replica.cc:330] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: stopping tablet replica
I20240501 08:00:54.373935 21337 raft_consensus.cc:2229] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:54.374096 21337 raft_consensus.cc:2258] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:54.374848 21337 ts_tablet_manager.cc:1875] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:54.375661 16563 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.376103 16693 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:54.380681 17220 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.382230 21337 ts_tablet_manager.cc:1888] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 08:00:54.382318 21337 log.cc:1196] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/fa0d919844a349549b1ad6ffbe440837
I20240501 08:00:54.383592 21252 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:54.383831 18838 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.384290 16405 catalog_manager.cc:4830] TS 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): tablet fa0d919844a349549b1ad6ffbe440837 (table rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]) successfully deleted
I20240501 08:00:54.384894 18969 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.391440 21499 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.391881 16562 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.394348 17089 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.394779 17220 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.395697 19097 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.398674 16414 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.401204 18838 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.408978 16693 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.412595 17353 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.415083 16825 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:54.417424 17089 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.417639 18838 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.418311 17354 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.418349 21252 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.418591 17220 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.418944 21300 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.419137 21252 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.419432 21252 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.422219 16693 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.422720 21252 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.423272 21252 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.423781 19097 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.424322 21300 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.424541 17353 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.426251 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6fd34b65ff134bf485af45d1eccd62f9 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:54.426143 19097 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.427868 17089 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.429625 16417 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a reported cstate change: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.430522 18838 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.432560 16541 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd34b65ff134bf485af45d1eccd62f9 with delete_type TABLET_DATA_TOMBSTONED (TS eca6f08d57d1453784f726199474a671 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:50112
I20240501 08:00:54.432807 21450 tablet_replica.cc:330] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671: stopping tablet replica
I20240501 08:00:54.432942 21450 raft_consensus.cc:2229] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:54.432832 17221 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.433174 21450 raft_consensus.cc:2258] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:54.433904 21450 ts_tablet_manager.cc:1875] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:54.436527 21450 ts_tablet_manager.cc:1888] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 08:00:54.436630 21450 log.cc:1196] T 6fd34b65ff134bf485af45d1eccd62f9 P eca6f08d57d1453784f726199474a671: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/6fd34b65ff134bf485af45d1eccd62f9
I20240501 08:00:54.437150 16405 catalog_manager.cc:4830] TS eca6f08d57d1453784f726199474a671 (127.0.99.129:43741): tablet 6fd34b65ff134bf485af45d1eccd62f9 (table rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]) successfully deleted
I20240501 08:00:54.438012 16693 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:54.468504 21538 ts_tablet_manager.cc:902] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987)
I20240501 08:00:54.469122 21538 tablet_copy_client.cc:308] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:41987
I20240501 08:00:54.474566 16979 tablet_copy_service.cc:140] P d5a55f6821d84c968a42f4a5c9ad07a7: Received BeginTabletCopySession request for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:35129)
I20240501 08:00:54.474689 16979 tablet_copy_service.cc:161] P d5a55f6821d84c968a42f4a5c9ad07a7: Beginning new tablet copy session on tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:35129: session id = 5f8132e5a8014a6f913ee9ac3338600a-d5ecdf2c2cdb4487b0e6315f63e1b1b8
I20240501 08:00:54.475669 16979 tablet_copy_source_session.cc:205] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:54.476773 21538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:54.484194 16825 consensus_queue.cc:237] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:54.486624 16562 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.486768 17353 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.487115 17221 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.487840 17091 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.487907 21498 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.488355 21498 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.489368 21501 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.491250 19097 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.491497 21252 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.492336 16693 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 08:00:54.493921 16761 consensus_peers.cc:485] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 99e2e8b8e2024846b0841efc429186ea. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:54.494122 21252 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.494488 21252 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.498541 21538 tablet_copy_client.cc:783] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:54.499491 21538 tablet_copy_client.cc:647] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:54.499416 17220 consensus_queue.cc:237] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:54.502009 16693 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.502235 21300 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:54.502614 18838 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.503150 21411 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.503402 17353 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.504302 16825 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.504284 21414 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.504601 21414 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.505331 18969 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.505831 17354 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:54.507608 19097 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:54.507894 16417 catalog_manager.cc:5484] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:54.508174 17091 raft_consensus.cc:1261] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:54.510564 21414 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.510677 21538 tablet_copy_client.cc:523] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:54.511970 17220 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:54.512590 21538 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:00:54.512631 21417 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.512557 21416 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.512924 21417 consensus_queue.cc:1035] T 7c25b2699528442f8e2993efbac96bbd P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:54.515046 16695 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.515276 17091 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.517088 18969 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.517728 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7c25b2699528442f8e2993efbac96bbd with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:54.518641 16417 catalog_manager.cc:5484] T 7c25b2699528442f8e2993efbac96bbd P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.521574 17089 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:54.521724 16825 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.523510 16541 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c25b2699528442f8e2993efbac96bbd with delete_type TABLET_DATA_TOMBSTONED (TS eca6f08d57d1453784f726199474a671 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:50112
I20240501 08:00:54.523794 21450 tablet_replica.cc:330] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671: stopping tablet replica
I20240501 08:00:54.523958 21450 raft_consensus.cc:2229] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:54.524204 21450 raft_consensus.cc:2258] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:54.524240 16693 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:54.524945 21450 ts_tablet_manager.cc:1875] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:54.526289 17354 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.527284 21450 ts_tablet_manager.cc:1888] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 08:00:54.527356 21450 log.cc:1196] T 7c25b2699528442f8e2993efbac96bbd P eca6f08d57d1453784f726199474a671: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/7c25b2699528442f8e2993efbac96bbd
I20240501 08:00:54.528915 16405 catalog_manager.cc:4830] TS eca6f08d57d1453784f726199474a671 (127.0.99.129:43741): tablet 7c25b2699528442f8e2993efbac96bbd (table rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]) successfully deleted
I20240501 08:00:54.530440 18839 raft_consensus.cc:2932] T 7c25b2699528442f8e2993efbac96bbd P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:54.533982 16562 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:00:54.549916 16562 consensus_queue.cc:237] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:54.552426 17354 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.552551 17089 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.554088 16825 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 08:00:54.554419 16497 consensus_peers.cc:485] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: addb23c607754d0aabee4c934f390251. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:54.554687 21428 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.555470 21428 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.555651 21430 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.556430 16959 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.556627 18969 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.557452 16693 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:00:54.557529 21429 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.558202 21427 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.560113 21299 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:00:54.565611 21538 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:54.565943 21538 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:00:54.566136 21538 ts_tablet_manager.cc:1367] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.054s user 0.020s sys 0.005s
I20240501 08:00:54.567998 21427 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.568670 21538 raft_consensus.cc:348] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:54.568815 21538 raft_consensus.cc:729] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:00:54.568924 21538 consensus_queue.cc:260] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:54.569247 21538 ts_tablet_manager.cc:1398] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240501 08:00:54.569693 17354 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.571169 16979 tablet_copy_service.cc:342] P d5a55f6821d84c968a42f4a5c9ad07a7: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-d5ecdf2c2cdb4487b0e6315f63e1b1b8 received from {username='slave'} at 127.0.99.138:35129
I20240501 08:00:54.571262 16979 tablet_copy_service.cc:433] P d5a55f6821d84c968a42f4a5c9ad07a7: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-d5ecdf2c2cdb4487b0e6315f63e1b1b8 on tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:00:54.572144 16693 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.572536 16416 catalog_manager.cc:5484] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:54.576390 16825 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.579968 18969 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.588974 16959 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.593014 17089 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:00:54.648535 16562 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00"
dest_uuid: "eca6f08d57d1453784f726199474a671"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58442
I20240501 08:00:54.648799 16562 raft_consensus.cc:595] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:54.698772 16562 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00"
dest_uuid: "eca6f08d57d1453784f726199474a671"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58470
I20240501 08:00:54.698972 16562 raft_consensus.cc:595] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:54.752600 21299 raft_consensus.cc:979] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: : Instructing follower 3d06b1f1890443da898fde2eb82daad3 to start an election
I20240501 08:00:54.752750 21299 raft_consensus.cc:1067] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Signalling peer 3d06b1f1890443da898fde2eb82daad3 to start an election
I20240501 08:00:54.753432 17220 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00"
dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
from {username='slave'} at 127.0.99.129:44341
I20240501 08:00:54.753610 17220 raft_consensus.cc:482] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:54.753657 17220 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:54.754706 17220 raft_consensus.cc:504] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:54.755517 17220 leader_election.cc:288] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 2 election: Requested vote from peers d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529), 873218458f2346348be2120e543990e2 (127.0.99.137:33839)
I20240501 08:00:54.755756 17089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:54.755892 17089 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:54.756249 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:54.756258 16693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:54.756382 16693 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:54.756379 16959 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:54.757108 17089 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 2.
I20240501 08:00:54.757164 16959 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 2.
I20240501 08:00:54.757382 16693 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 2.
I20240501 08:00:54.757982 16562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:54.758126 16562 raft_consensus.cc:3032] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:00:54.758177 16562 raft_consensus.cc:729] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Running, Role: LEADER
I20240501 08:00:54.758249 18969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "873218458f2346348be2120e543990e2"
I20240501 08:00:54.758352 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:54.758378 16562 consensus_queue.cc:260] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:54.758469 16825 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:54.758452 18969 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:54.758678 16562 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:54.759562 16825 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 2.
I20240501 08:00:54.759603 16562 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 2.
I20240501 08:00:54.759634 18969 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 2.
I20240501 08:00:54.760061 17156 leader_election.cc:302] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 8 voters: 5 yes votes; 0 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, 64196487691e4623b690b2869aad57f7, 873218458f2346348be2120e543990e2, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:54.760294 21417 raft_consensus.cc:2781] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:00:54.760377 21417 raft_consensus.cc:686] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 2 LEADER]: Becoming Leader. State: Replica: 3d06b1f1890443da898fde2eb82daad3, State: Running, Role: LEADER
I20240501 08:00:54.760558 21417 consensus_queue.cc:237] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:54.762593 19097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8a14c2447e45c5b148ff762bb55c00" candidate_uuid: "3d06b1f1890443da898fde2eb82daad3" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "5f8132e5a8014a6f913ee9ac3338600a"
I20240501 08:00:54.762799 19097 raft_consensus.cc:3037] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:54.763873 19097 raft_consensus.cc:2454] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d06b1f1890443da898fde2eb82daad3 in term 2.
I20240501 08:00:54.764691 16417 catalog_manager.cc:5484] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: term changed from 1 to 2, leader changed from eca6f08d57d1453784f726199474a671 (127.0.99.129) to 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134). New cstate: current_term: 2 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:54.894194 21636 ts_tablet_manager.cc:902] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:54.894699 21636 tablet_copy_client.cc:308] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:43779
I20240501 08:00:54.895206 16847 tablet_copy_service.cc:140] P 1160c6e72055456992b5d8bf5f3b45c0: Received BeginTabletCopySession request for tablet 99e2e8b8e2024846b0841efc429186ea from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:35009)
I20240501 08:00:54.895315 16847 tablet_copy_service.cc:161] P 1160c6e72055456992b5d8bf5f3b45c0: Beginning new tablet copy session on tablet 99e2e8b8e2024846b0841efc429186ea from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:35009: session id = 5aaf071b45ab469cb23321d37fad37f8-99e2e8b8e2024846b0841efc429186ea
I20240501 08:00:54.896095 16847 tablet_copy_source_session.cc:205] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:54.897011 21636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99e2e8b8e2024846b0841efc429186ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:54.898876 21636 tablet_copy_client.cc:783] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:54.899037 21636 tablet_copy_client.cc:647] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:54.900306 21636 tablet_copy_client.cc:523] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:54.901995 21636 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:00:54.925994 21636 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:54.926321 21636 tablet_bootstrap.cc:492] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:00:54.926504 21636 ts_tablet_manager.cc:1367] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.025s user 0.018s sys 0.006s
I20240501 08:00:54.926927 21636 raft_consensus.cc:348] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:54.927071 21636 raft_consensus.cc:729] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:00:54.927182 21636 consensus_queue.cc:260] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:00:54.927464 21636 ts_tablet_manager.cc:1398] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:00:54.928476 16847 tablet_copy_service.cc:342] P 1160c6e72055456992b5d8bf5f3b45c0: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-99e2e8b8e2024846b0841efc429186ea received from {username='slave'} at 127.0.99.136:35009
I20240501 08:00:54.928565 16847 tablet_copy_service.cc:433] P 1160c6e72055456992b5d8bf5f3b45c0: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-99e2e8b8e2024846b0841efc429186ea on tablet 99e2e8b8e2024846b0841efc429186ea with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:00:54.933940 19097 raft_consensus.cc:1203] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:54.938290 21538 ts_tablet_manager.cc:902] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:00:54.938834 21538 tablet_copy_client.cc:308] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:43741
I20240501 08:00:54.939327 16583 tablet_copy_service.cc:140] P eca6f08d57d1453784f726199474a671: Received BeginTabletCopySession request for tablet addb23c607754d0aabee4c934f390251 from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:34739)
I20240501 08:00:54.939435 16583 tablet_copy_service.cc:161] P eca6f08d57d1453784f726199474a671: Beginning new tablet copy session on tablet addb23c607754d0aabee4c934f390251 from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:34739: session id = 5f8132e5a8014a6f913ee9ac3338600a-addb23c607754d0aabee4c934f390251
I20240501 08:00:54.940220 16583 tablet_copy_source_session.cc:205] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:54.941145 21538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addb23c607754d0aabee4c934f390251. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:54.943174 21538 tablet_copy_client.cc:783] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:54.943344 21538 tablet_copy_client.cc:647] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:54.944533 21538 tablet_copy_client.cc:523] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:54.945748 21538 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:00:54.967110 21538 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:54.967444 21538 tablet_bootstrap.cc:492] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:00:54.967641 21538 ts_tablet_manager.cc:1367] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.000s
I20240501 08:00:54.968168 21538 raft_consensus.cc:348] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:54.968297 21538 raft_consensus.cc:729] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:00:54.968394 21538 consensus_queue.cc:260] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:00:54.968667 21538 ts_tablet_manager.cc:1398] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 08:00:54.969203 16583 tablet_copy_service.cc:342] P eca6f08d57d1453784f726199474a671: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-addb23c607754d0aabee4c934f390251 received from {username='slave'} at 127.0.99.138:34739
I20240501 08:00:54.969285 16583 tablet_copy_service.cc:433] P eca6f08d57d1453784f726199474a671: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-addb23c607754d0aabee4c934f390251 on tablet addb23c607754d0aabee4c934f390251 with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:00:55.144796 16825 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:55.145471 21417 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.207820 16959 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:55.208384 21417 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.211989 16693 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:55.212390 21417 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.242800 16562 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:55.243258 21417 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.245779 17089 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:55.246376 21691 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.248047 18969 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:55.251062 21693 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.257393 19097 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:55.258831 21414 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.307655 18838 raft_consensus.cc:1203] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:55.403743 19097 raft_consensus.cc:1203] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:00:55.431825 21662 raft_consensus.cc:1050] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:00:55.432454 21662 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:55.434518 18838 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.434769 17354 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.435206 16693 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.435290 17220 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.435427 19097 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.435678 21729 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.436189 21727 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.436539 21729 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.436578 16562 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.436674 16825 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.437378 21662 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.437713 21662 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.438006 21662 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.438304 21662 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.440738 21728 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.441342 16825 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.441830 16695 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.444247 17353 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.444252 16562 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.444773 16416 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.446146 19097 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.446707 18839 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.450726 17220 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.456718 16959 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:55.458498 16693 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:55.458576 17353 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:55.458986 21727 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.459039 18839 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:55.459398 21727 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.459569 21728 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.464991 21727 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.465401 17220 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:55.465860 19097 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:55.465888 21728 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.466682 16562 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:55.467314 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:55.467355 21242 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.467350 17220 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.467686 17353 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.468155 18839 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.468605 19097 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.471012 21662 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.471431 16413 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a reported cstate change: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.474609 16562 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.475221 16807 tablet_service.cc:1514] Processing DeleteTablet for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with delete_type TABLET_DATA_TOMBSTONED (TS 1160c6e72055456992b5d8bf5f3b45c0 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:39058
I20240501 08:00:55.475821 16693 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.478554 21740 tablet_replica.cc:330] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: stopping tablet replica
I20240501 08:00:55.478767 21740 raft_consensus.cc:2229] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:55.479060 21740 raft_consensus.cc:2258] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:55.479862 21740 ts_tablet_manager.cc:1875] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:55.482458 21740 ts_tablet_manager.cc:1888] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 08:00:55.482667 21740 log.cc:1196] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/d5ecdf2c2cdb4487b0e6315f63e1b1b8
I20240501 08:00:55.483464 16405 catalog_manager.cc:4830] TS 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 (table rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]) successfully deleted
I20240501 08:00:55.759150 17220 consensus_queue.cc:237] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 9, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:55.761195 19102 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 08:00:55.761366 18969 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 08:00:55.761793 16693 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 08:00:55.761849 21417 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:55.762166 16825 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 08:00:55.762235 21417 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:55.762638 16959 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 08:00:55.763036 21690 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:55.763401 21690 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:55.763816 21693 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:55.765141 17089 raft_consensus.cc:1261] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 08:00:55.765622 21417 consensus_queue.cc:1035] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:55.765444 21690 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:55.766707 17089 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:55.767266 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4b8a14c2447e45c5b148ff762bb55c00 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:55.768126 16825 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 1160c6e72055456992b5d8bf5f3b45c0 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:55.768600 18969 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:55.770414 16959 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:55.770635 16693 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 64196487691e4623b690b2869aad57f7 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:55.770723 19102 raft_consensus.cc:2932] T 4b8a14c2447e45c5b148ff762bb55c00 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:55.771203 16414 catalog_manager.cc:5484] T 4b8a14c2447e45c5b148ff762bb55c00 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:55.775095 16541 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b8a14c2447e45c5b148ff762bb55c00 with delete_type TABLET_DATA_TOMBSTONED (TS eca6f08d57d1453784f726199474a671 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:50112
I20240501 08:00:55.775573 21777 tablet_replica.cc:330] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: stopping tablet replica
I20240501 08:00:55.775746 21777 raft_consensus.cc:2229] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:55.775960 21777 raft_consensus.cc:2258] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:55.776702 21777 ts_tablet_manager.cc:1875] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:55.779012 21777 ts_tablet_manager.cc:1888] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.10
I20240501 08:00:55.779096 21777 log.cc:1196] T 4b8a14c2447e45c5b148ff762bb55c00 P eca6f08d57d1453784f726199474a671: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/4b8a14c2447e45c5b148ff762bb55c00
I20240501 08:00:55.779636 16405 catalog_manager.cc:4830] TS eca6f08d57d1453784f726199474a671 (127.0.99.129:43741): tablet 4b8a14c2447e45c5b148ff762bb55c00 (table rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]) successfully deleted
I20240501 08:00:55.801267 21498 raft_consensus.cc:1050] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:00:55.802256 21498 consensus_queue.cc:237] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:55.804816 16693 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.804941 17353 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.805181 17089 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.805263 17220 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.805377 21502 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.805629 21498 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.805692 16562 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.805805 18839 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.806110 19102 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.806124 20953 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.806352 21780 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.806648 21498 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.806973 21498 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.807255 21498 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.810403 21780 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.810875 17353 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.811511 16693 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.812371 17089 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.813915 16562 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.814879 16420 catalog_manager.cc:5484] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:55.815758 17220 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.816020 18839 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.817004 19102 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.821110 16825 consensus_queue.cc:237] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } }
I20240501 08:00:55.823132 17089 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:55.823302 17354 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:55.823340 17220 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:55.823896 20953 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.824142 21780 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.824240 16693 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:55.824268 19102 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:55.824335 21779 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.824729 21502 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.824796 18839 raft_consensus.cc:1261] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:00:55.824864 20953 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.825443 21502 consensus_queue.cc:1035] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:55.827479 21502 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.827857 17220 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.828603 19097 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.828943 18839 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.830886 16414 catalog_manager.cc:5484] T 99e2e8b8e2024846b0841efc429186ea P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.831420 17089 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.831610 17354 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.832276 16693 raft_consensus.cc:2932] T 99e2e8b8e2024846b0841efc429186ea P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } }
I20240501 08:00:55.831104 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 99e2e8b8e2024846b0841efc429186ea with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:55.835564 16541 tablet_service.cc:1514] Processing DeleteTablet for tablet 99e2e8b8e2024846b0841efc429186ea with delete_type TABLET_DATA_TOMBSTONED (TS eca6f08d57d1453784f726199474a671 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:50112
I20240501 08:00:55.835845 21777 tablet_replica.cc:330] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671: stopping tablet replica
I20240501 08:00:55.836014 21777 raft_consensus.cc:2229] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:55.836292 21777 raft_consensus.cc:2258] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:55.837059 21777 ts_tablet_manager.cc:1875] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:55.839412 21777 ts_tablet_manager.cc:1888] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 08:00:55.839531 21777 log.cc:1196] T 99e2e8b8e2024846b0841efc429186ea P eca6f08d57d1453784f726199474a671: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/99e2e8b8e2024846b0841efc429186ea
I20240501 08:00:55.840088 16405 catalog_manager.cc:4830] TS eca6f08d57d1453784f726199474a671 (127.0.99.129:43741): tablet 99e2e8b8e2024846b0841efc429186ea (table rebalance_test_table_6 [id=9783b8873ef149a6a8347817c8d3d859]) successfully deleted
I20240501 08:00:55.969602 21251 raft_consensus.cc:1050] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:00:55.970157 21251 consensus_queue.cc:237] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:55.971988 17089 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.972195 18969 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.972270 19102 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.972507 21816 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.972612 16693 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.972779 17354 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.972926 16959 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.972991 21812 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.973469 21816 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.973623 16825 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:00:55.973654 21814 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.973809 21251 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.973937 21815 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.974303 21814 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:00:55.977365 19102 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.976889 21816 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.978173 16959 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.978730 16693 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.979830 17089 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.980939 16825 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.981709 16419 catalog_manager.cc:5484] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:00:55.982678 17354 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:55.985886 18969 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:56.087664 16562 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "addb23c607754d0aabee4c934f390251"
dest_uuid: "eca6f08d57d1453784f726199474a671"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49194
I20240501 08:00:56.087846 16562 raft_consensus.cc:595] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:56.302466 16562 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "addb23c607754d0aabee4c934f390251"
dest_uuid: "eca6f08d57d1453784f726199474a671"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:49212
I20240501 08:00:56.302623 16562 raft_consensus.cc:595] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:56.386221 21251 raft_consensus.cc:979] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: : Instructing follower 1160c6e72055456992b5d8bf5f3b45c0 to start an election
I20240501 08:00:56.386364 21251 raft_consensus.cc:1067] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Signalling peer 1160c6e72055456992b5d8bf5f3b45c0 to start an election
I20240501 08:00:56.386946 16825 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "addb23c607754d0aabee4c934f390251"
dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
from {username='slave'} at 127.0.99.129:60079
I20240501 08:00:56.387156 16825 raft_consensus.cc:482] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:56.387217 16825 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:56.388375 16825 raft_consensus.cc:504] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:56.389226 16825 leader_election.cc:288] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 2 election: Requested vote from peers eca6f08d57d1453784f726199474a671 (127.0.99.129:43741), 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219), d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), 873218458f2346348be2120e543990e2 (127.0.99.137:33839), 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529)
I20240501 08:00:56.389465 16695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "64196487691e4623b690b2869aad57f7"
I20240501 08:00:56.389598 16695 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:56.389921 16959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7"
I20240501 08:00:56.390045 16959 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:56.390100 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:56.390208 17354 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:56.390766 16695 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 2.
I20240501 08:00:56.390980 17354 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 2.
I20240501 08:00:56.391076 16959 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 2.
I20240501 08:00:56.391356 19102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "5f8132e5a8014a6f913ee9ac3338600a"
I20240501 08:00:56.391503 19102 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:56.391657 17089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:56.391793 17089 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:56.391867 18969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "873218458f2346348be2120e543990e2"
I20240501 08:00:56.392014 18969 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:56.392024 16562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addb23c607754d0aabee4c934f390251" candidate_uuid: "1160c6e72055456992b5d8bf5f3b45c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "eca6f08d57d1453784f726199474a671"
I20240501 08:00:56.392145 16562 raft_consensus.cc:3032] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:00:56.392181 16562 raft_consensus.cc:729] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Running, Role: LEADER
I20240501 08:00:56.392375 16562 consensus_queue.cc:260] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:56.392585 19102 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 2.
I20240501 08:00:56.392697 16562 raft_consensus.cc:3037] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:56.392907 17089 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 2.
I20240501 08:00:56.393039 18969 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 2.
I20240501 08:00:56.393299 16763 leader_election.cc:302] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 8 voters: 5 yes votes; 0 no votes. yes voters: 1160c6e72055456992b5d8bf5f3b45c0, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, d5a55f6821d84c968a42f4a5c9ad07a7, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:56.393731 16562 raft_consensus.cc:2454] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1160c6e72055456992b5d8bf5f3b45c0 in term 2.
I20240501 08:00:56.393965 21780 raft_consensus.cc:2781] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:00:56.394062 21780 raft_consensus.cc:686] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 2 LEADER]: Becoming Leader. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Running, Role: LEADER
I20240501 08:00:56.394243 21780 consensus_queue.cc:237] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:56.397228 16414 catalog_manager.cc:5484] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: term changed from 1 to 2, leader changed from eca6f08d57d1453784f726199474a671 (127.0.99.129) to 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131). New cstate: current_term: 2 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:56.829675 17089 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:56.830130 21780 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:56.837554 16695 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:56.838218 21780 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:56.852399 19102 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:56.852838 21780 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:56.883688 17354 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:56.884150 21780 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:56.888339 16562 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:56.889243 16959 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:56.890585 18969 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 08:00:56.891639 21498 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:56.891844 21860 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:56.894400 21862 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:00:57.379884 16825 consensus_queue.cc:237] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 9, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:00:57.381858 17354 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 08:00:57.381899 18969 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 08:00:57.382154 17091 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 08:00:57.382150 16693 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 08:00:57.382306 21498 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:57.382503 16957 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 08:00:57.382617 21861 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:57.382766 21860 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:57.382956 21498 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:57.383219 19102 raft_consensus.cc:1261] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 08:00:57.383424 21780 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:57.383703 21861 consensus_queue.cc:1035] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:57.386261 21860 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 1160c6e72055456992b5d8bf5f3b45c0 [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:57.387466 17354 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:57.387764 18969 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:57.389892 16693 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 64196487691e4623b690b2869aad57f7 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:57.390437 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet addb23c607754d0aabee4c934f390251 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:00:57.390419 19102 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:57.390580 16417 catalog_manager.cc:5484] T addb23c607754d0aabee4c934f390251 P 873218458f2346348be2120e543990e2 reported cstate change: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:57.391613 17091 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:57.394163 16957 raft_consensus.cc:2932] T addb23c607754d0aabee4c934f390251 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:00:57.396346 16541 tablet_service.cc:1514] Processing DeleteTablet for tablet addb23c607754d0aabee4c934f390251 with delete_type TABLET_DATA_TOMBSTONED (TS eca6f08d57d1453784f726199474a671 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:50112
I20240501 08:00:57.397832 21892 tablet_replica.cc:330] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: stopping tablet replica
I20240501 08:00:57.398087 21892 raft_consensus.cc:2229] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:00:57.398241 21892 raft_consensus.cc:2258] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:00:57.399070 21892 ts_tablet_manager.cc:1875] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:00:57.401772 21892 ts_tablet_manager.cc:1888] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.10
I20240501 08:00:57.401855 21892 log.cc:1196] T addb23c607754d0aabee4c934f390251 P eca6f08d57d1453784f726199474a671: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/addb23c607754d0aabee4c934f390251
I20240501 08:00:57.402415 16405 catalog_manager.cc:4830] TS eca6f08d57d1453784f726199474a671 (127.0.99.129:43741): tablet addb23c607754d0aabee4c934f390251 (table rebalance_test_table_1 [id=40d72190d5b94c8e877576025ac91920]) successfully deleted
I20240501 08:00:57.633332 19076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:57.633456 18818 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:57.635092 17201 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 08:00:57.648161 16807 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 08:00:57.653774 16674 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:57.659878 17071 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:57.663017 16939 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:57.674764 17335 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:00:57.683413 18950 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:00:57.719169 16541 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:00:57.760306 16825 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:57.762542 17220 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:00:57.762778 19102 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:00:57.762835 17353 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240501 08:00:57.763196 16760 consensus_peers.cc:485] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6fd34b65ff134bf485af45d1eccd62f9. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:00:57.763285 16693 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:00:57.763484 17089 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:00:57.763510 21860 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:57.763770 18839 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:00:57.763864 21950 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:57.764118 21860 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:57.764477 21860 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:57.764637 21861 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:57.764966 21861 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:00:57.767304 21862 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:57.769378 17353 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:57.769737 17220 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:57.770490 17089 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:57.770784 16420 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:57.773689 16693 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:57.774562 18839 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:57.776235 19102 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:00:58.299361 21974 ts_tablet_manager.cc:902] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:00:58.299957 21974 tablet_copy_client.cc:308] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:43779
I20240501 08:00:58.300494 16847 tablet_copy_service.cc:140] P 1160c6e72055456992b5d8bf5f3b45c0: Received BeginTabletCopySession request for tablet 6fd34b65ff134bf485af45d1eccd62f9 from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:39189)
I20240501 08:00:58.300619 16847 tablet_copy_service.cc:161] P 1160c6e72055456992b5d8bf5f3b45c0: Beginning new tablet copy session on tablet 6fd34b65ff134bf485af45d1eccd62f9 from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:39189: session id = 873218458f2346348be2120e543990e2-6fd34b65ff134bf485af45d1eccd62f9
I20240501 08:00:58.301426 16847 tablet_copy_source_session.cc:205] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:00:58.302434 21974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd34b65ff134bf485af45d1eccd62f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:00:58.304677 21974 tablet_copy_client.cc:783] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:00:58.304828 21974 tablet_copy_client.cc:647] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:00:58.306156 21974 tablet_copy_client.cc:523] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:00:58.307701 21974 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:00:58.343118 21974 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:00:58.343528 21974 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:00:58.343736 21974 ts_tablet_manager.cc:1367] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.036s user 0.021s sys 0.000s
I20240501 08:00:58.344244 21974 raft_consensus.cc:348] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:58.344393 21974 raft_consensus.cc:729] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:00:58.344491 21974 consensus_queue.cc:260] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:00:58.344895 21974 ts_tablet_manager.cc:1398] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240501 08:00:58.345443 16847 tablet_copy_service.cc:342] P 1160c6e72055456992b5d8bf5f3b45c0: Request end of tablet copy session 873218458f2346348be2120e543990e2-6fd34b65ff134bf485af45d1eccd62f9 received from {username='slave'} at 127.0.99.137:39189
I20240501 08:00:58.345528 16847 tablet_copy_service.cc:433] P 1160c6e72055456992b5d8bf5f3b45c0: ending tablet copy session 873218458f2346348be2120e543990e2-6fd34b65ff134bf485af45d1eccd62f9 on tablet 6fd34b65ff134bf485af45d1eccd62f9 with peer 873218458f2346348be2120e543990e2
I20240501 08:00:58.705140 18969 raft_consensus.cc:1203] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 08:00:59.210549 21498 raft_consensus.cc:1050] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:00:59.211213 21498 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:59.213557 18839 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:00:59.214155 19102 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:00:59.214155 17089 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:00:59.214339 22010 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:00:59.214536 18970 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:00:59.214553 22007 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:00:59.214740 16695 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:00:59.214948 22010 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:00:59.215322 22008 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:00:59.215584 17354 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:00:59.215654 17222 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:00:59.216041 22011 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:00:59.217139 22011 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:00:59.217633 22007 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:00:59.219717 22009 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:59.221378 17354 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:59.222029 19097 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:59.222386 18839 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:59.224654 18970 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:59.225715 16415 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 11 to 12, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:59.226676 17089 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:59.227142 16693 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:59.232103 17222 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:00:59.253681 16825 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9"
dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55380
I20240501 08:00:59.253868 16825 raft_consensus.cc:595] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Received request to transfer leadership
W20240501 08:00:59.390259 16868 debug-util.cc:398] Leaking SignalData structure 0x555c0c8d6980 after lost signal to thread 21821
I20240501 08:00:59.475565 16825 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9"
dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:55400
I20240501 08:00:59.475737 16825 raft_consensus.cc:595] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Received request to transfer leadership
I20240501 08:00:59.653718 22012 raft_consensus.cc:979] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: : Instructing follower 64196487691e4623b690b2869aad57f7 to start an election
I20240501 08:00:59.653873 22008 raft_consensus.cc:1067] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Signalling peer 64196487691e4623b690b2869aad57f7 to start an election
I20240501 08:00:59.654525 16695 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9"
dest_uuid: "64196487691e4623b690b2869aad57f7"
from {username='slave'} at 127.0.99.131:47969
I20240501 08:00:59.654702 16695 raft_consensus.cc:482] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 08:00:59.654757 16695 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:59.655699 16695 raft_consensus.cc:504] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:59.656481 16695 leader_election.cc:288] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779), 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813), e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463), af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065), 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529), 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371), 873218458f2346348be2120e543990e2 (127.0.99.137:33839)
I20240501 08:00:59.656738 17354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "af70005d0c6c48bc9919cae6346f221b"
I20240501 08:00:59.656738 17089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "e282199dc2fa4977b83ff07b86cb0690"
I20240501 08:00:59.656889 17089 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:59.656889 17354 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:59.657536 19097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "5f8132e5a8014a6f913ee9ac3338600a"
I20240501 08:00:59.657667 19097 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:59.658016 17354 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 2.
I20240501 08:00:59.658016 17089 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 2.
I20240501 08:00:59.658371 17222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "3d06b1f1890443da898fde2eb82daad3"
I20240501 08:00:59.658516 17222 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:59.658529 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "1160c6e72055456992b5d8bf5f3b45c0"
I20240501 08:00:59.658584 19097 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 2.
I20240501 08:00:59.658650 16825 raft_consensus.cc:3032] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 08:00:59.658721 16825 raft_consensus.cc:729] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Running, Role: LEADER
I20240501 08:00:59.658932 16825 consensus_queue.cc:260] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:59.659178 18839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "5aaf071b45ab469cb23321d37fad37f8"
I20240501 08:00:59.659245 16825 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:59.659391 18839 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:59.659560 17222 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 2.
I20240501 08:00:59.660181 16630 leader_election.cc:302] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 8 voters: 5 yes votes; 0 no votes. yes voters: 3d06b1f1890443da898fde2eb82daad3, 5f8132e5a8014a6f913ee9ac3338600a, 64196487691e4623b690b2869aad57f7, af70005d0c6c48bc9919cae6346f221b, e282199dc2fa4977b83ff07b86cb0690; no voters:
I20240501 08:00:59.660359 16825 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 2.
I20240501 08:00:59.660576 18839 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 2.
I20240501 08:00:59.660725 21309 raft_consensus.cc:2781] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 08:00:59.660841 21309 raft_consensus.cc:686] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 2 LEADER]: Becoming Leader. State: Replica: 64196487691e4623b690b2869aad57f7, State: Running, Role: LEADER
I20240501 08:00:59.661020 21309 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:00:59.663758 16420 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 reported cstate change: term changed from 1 to 2, leader changed from 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) to 64196487691e4623b690b2869aad57f7 (127.0.99.130). New cstate: current_term: 2 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:00:59.664651 18970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd34b65ff134bf485af45d1eccd62f9" candidate_uuid: "64196487691e4623b690b2869aad57f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: true dest_uuid: "873218458f2346348be2120e543990e2"
I20240501 08:00:59.664860 18970 raft_consensus.cc:3037] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Advancing to term 2
I20240501 08:00:59.666276 18970 raft_consensus.cc:2454] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64196487691e4623b690b2869aad57f7 in term 2.
I20240501 08:01:00.081440 16825 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240501 08:01:00.081939 21309 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:00.120771 17222 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240501 08:01:00.121318 21309 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:00.173010 18839 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240501 08:01:00.173527 21309 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:00.193703 17354 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240501 08:01:00.194204 21309 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:00.197677 19097 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240501 08:01:00.197822 18970 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240501 08:01:00.198217 22071 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:00.198884 22071 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:00.203998 17089 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240501 08:01:00.204996 22078 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:00.752223 16695 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 2.13, Last appended by leader: 12, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } }
I20240501 08:01:00.754231 18970 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20240501 08:01:00.754374 17354 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20240501 08:01:00.754374 17220 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20240501 08:01:00.754573 17089 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20240501 08:01:00.754969 19097 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20240501 08:01:00.755010 18839 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 13. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20240501 08:01:00.755007 22108 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:00.755395 21309 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:00.755578 22109 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:00.755738 22108 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:00.755900 22107 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:00.756418 22071 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:00.765026 22109 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 2 LEADER]: Committing config change with OpId 2.14: config changed from index 12 to 14, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:01:00.766173 18970 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Committing config change with OpId 2.14: config changed from index 12 to 14, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:01:00.766304 17220 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 2 FOLLOWER]: Committing config change with OpId 2.14: config changed from index 12 to 14, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:01:00.767663 18839 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 2 FOLLOWER]: Committing config change with OpId 2.14: config changed from index 12 to 14, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:01:00.769239 19102 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Committing config change with OpId 2.14: config changed from index 12 to 14, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:01:00.770334 16418 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 reported cstate change: config changed from index 12 to 14, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New cstate: current_term: 2 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:01:00.771870 17089 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Committing config change with OpId 2.14: config changed from index 12 to 14, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:01:00.772532 16406 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6fd34b65ff134bf485af45d1eccd62f9 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:01:00.773017 17354 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Committing config change with OpId 2.14: config changed from index 12 to 14, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } }
I20240501 08:01:00.776440 16807 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd34b65ff134bf485af45d1eccd62f9 with delete_type TABLET_DATA_TOMBSTONED (TS 1160c6e72055456992b5d8bf5f3b45c0 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:39058
I20240501 08:01:00.776850 22117 tablet_replica.cc:330] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: stopping tablet replica
I20240501 08:01:00.776996 22117 raft_consensus.cc:2229] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:01:00.777168 22117 raft_consensus.cc:2258] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:01:00.777978 22117 ts_tablet_manager.cc:1875] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:01:00.780895 22117 ts_tablet_manager.cc:1888] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.13
I20240501 08:01:00.780985 22117 log.cc:1196] T 6fd34b65ff134bf485af45d1eccd62f9 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/6fd34b65ff134bf485af45d1eccd62f9
I20240501 08:01:00.781549 16405 catalog_manager.cc:4830] TS 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): tablet 6fd34b65ff134bf485af45d1eccd62f9 (table rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]) successfully deleted
I20240501 08:01:01.009632 19076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:01:01.010119 16807 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:01:01.010752 18818 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:01:01.020599 17201 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 08:01:01.024649 16674 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 08:01:01.036628 17071 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:01:01.036991 17335 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:01:01.049438 18950 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:01:01.069001 16939 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:01:01.071882 16541 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:01:01.223901 19076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:01:01.227903 16807 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 08:01:01.236387 18818 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:01:01.239907 16939 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:01:01.241272 17335 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:01:01.241829 18950 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 08:01:01.245952 16674 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 08:01:01.267954 16541 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 08:01:01.277477 17201 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 08:01:01.294641 17071 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 08:01:01.346755 16695 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 12, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:01.348969 17220 consensus_queue.cc:237] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } }
I20240501 08:01:01.349399 17354 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 2 index: 15. (index mismatch)
I20240501 08:01:01.349436 18839 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 2 index: 15. (index mismatch)
I20240501 08:01:01.349809 19102 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 2 index: 15. (index mismatch)
W20240501 08:01:01.349999 16630 consensus_peers.cc:485] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 -> Peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): Couldn't send request to peer d5a55f6821d84c968a42f4a5c9ad07a7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.350124 22227 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:01.350404 18970 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 2 index: 15. (index mismatch)
I20240501 08:01:01.350984 22109 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:01.351359 22109 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:01.351653 22109 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:01.352746 17354 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.353638 21973 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.353986 17220 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 2 index: 15. (index mismatch)
I20240501 08:01:01.354684 16562 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.355012 17091 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 2 index: 15. (index mismatch)
I20240501 08:01:01.355290 18970 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.355355 16825 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.355544 16695 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.355960 22245 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:01.356190 22227 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 2 LEADER]: Committing config change with OpId 2.15: config changed from index 14 to 15, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.356568 17089 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.356683 18839 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 2 FOLLOWER]: Committing config change with OpId 2.15: config changed from index 14 to 15, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.357470 19102 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Committing config change with OpId 2.15: config changed from index 14 to 15, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.355175 22247 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.359036 17089 consensus_queue.cc:237] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } }
W20240501 08:01:01.359660 17157 consensus_peers.cc:485] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d27d6c2cc194773a648909806f4416b. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.359582 18970 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Committing config change with OpId 2.15: config changed from index 14 to 15, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.359681 17353 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Committing config change with OpId 2.15: config changed from index 14 to 15, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.359877 21973 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.360277 21973 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.360560 21973 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.362435 16693 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.363003 17091 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Committing config change with OpId 2.15: config changed from index 14 to 15, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.363310 16416 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 14 to 15, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New cstate: current_term: 2 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.365784 22244 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:01.366101 21850 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240501 08:01:01.366798 17024 consensus_peers.cc:485] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 -> Peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741): Couldn't send request to peer eca6f08d57d1453784f726199474a671. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.366906 16825 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.367291 21850 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.368731 17222 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.368813 17220 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 2 FOLLOWER]: Committing config change with OpId 2.15: config changed from index 14 to 15, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.370013 19102 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.370453 22118 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.370829 17353 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.370872 22118 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.371201 22248 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.371357 18970 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.373936 21850 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.375577 22248 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:01:01.376322 21850 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.378717 17353 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:01:01.379261 16695 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:01:01.380681 18969 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:01:01.383414 16419 catalog_manager.cc:5484] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 reported cstate change: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:01:01.387450 16959 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } }
I20240501 08:01:01.389051 22262 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.389362 16825 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:01:01.390503 16563 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.390895 16826 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.391294 22101 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.391431 16693 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.392119 16694 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.392151 18839 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.392637 19097 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.393942 22101 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.394361 22101 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.394464 16562 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:01:01.394668 22050 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.396461 17222 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.397248 17353 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
W20240501 08:01:01.399241 16895 consensus_peers.cc:485] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 -> Peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): Couldn't send request to peer e282199dc2fa4977b83ff07b86cb0690. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.399341 18969 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.399986 17222 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.401315 19102 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.401938 16418 catalog_manager.cc:5484] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.402508 17354 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.403425 22050 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.404572 17091 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } }
I20240501 08:01:01.406153 22050 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.411959 22292 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.414476 18839 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.417351 16415 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 reported cstate change: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.420799 19097 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.424294 17091 consensus_queue.cc:237] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } }
I20240501 08:01:01.424361 17354 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.427202 17220 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.427553 16826 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240501 08:01:01.427798 17024 consensus_peers.cc:485] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 -> Peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741): Couldn't send request to peer eca6f08d57d1453784f726199474a671. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.427961 22260 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.428161 22118 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.428814 19097 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.428879 16693 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.429217 18839 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.429606 17354 raft_consensus.cc:1261] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.430019 22260 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.430204 22259 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.430371 22118 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.430505 22259 consensus_queue.cc:1035] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.432683 16563 consensus_queue.cc:237] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:01.433761 16959 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:01:01.436257 17221 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.436316 18838 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.436255 22260 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.437933 19097 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.438006 16693 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.438103 17353 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.438887 17091 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.439074 22192 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.439589 17220 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.440681 16694 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.440812 19102 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.441160 17353 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.441625 19097 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.441794 16415 catalog_manager.cc:5484] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:01:01.442251 22050 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.442530 18838 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.442714 22050 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.442778 16693 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.442976 22101 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.444583 16562 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.444830 21988 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.445238 21988 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.445539 21988 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.445823 21988 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.446113 21988 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240501 08:01:01.446713 16895 consensus_peers.cc:485] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 -> Peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): Couldn't send request to peer e282199dc2fa4977b83ff07b86cb0690. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.448323 16694 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.448575 16826 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.450670 17354 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
W20240501 08:01:01.452874 16498 consensus_peers.cc:485] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 -> Peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): Couldn't send request to peer d5a55f6821d84c968a42f4a5c9ad07a7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.453958 18839 raft_consensus.cc:2932] T 5e0b477de25646eda4f9f0e4921ec118 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } } }
I20240501 08:01:01.456866 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.457458 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.461817 16563 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.464574 22192 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.464720 22293 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.465428 18838 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
W20240501 08:01:01.466491 16892 consensus_peers.cc:485] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5ecdf2c2cdb4487b0e6315f63e1b1b8. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.467537 17353 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.467589 17220 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.467681 17091 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.470839 16693 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.471575 16414 catalog_manager.cc:5484] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 10 to 11, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.472167 16415 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 reported cstate change: config changed from index 11 to 12, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.475009 19102 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.475419 19097 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.479557 16563 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.483819 17221 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.484694 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.485445 16695 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.487242 17221 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.488036 18839 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.489045 17354 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.490238 17221 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.497097 17354 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } }
I20240501 08:01:01.499697 18969 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.500164 16959 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.500236 21790 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.500640 16563 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.500986 16695 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.501057 21790 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.501271 17221 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240501 08:01:01.501732 17288 consensus_peers.cc:485] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b -> Peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): Couldn't send request to peer 1160c6e72055456992b5d8bf5f3b45c0. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.502128 22196 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.502476 22196 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.503032 22196 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.503875 18839 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.504369 22196 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.506640 21790 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.507079 18839 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.509776 16418 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 10 to 11, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:01:01.509954 17222 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.510139 16563 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.514448 16695 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.519882 16959 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.520354 18969 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.528838 17353 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } }
I20240501 08:01:01.530956 18969 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
W20240501 08:01:01.531059 17291 consensus_peers.cc:485] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b -> Peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): Couldn't send request to peer e282199dc2fa4977b83ff07b86cb0690. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.531430 22349 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20240501 08:01:01.531582 17288 consensus_peers.cc:485] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b -> Peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): Couldn't send request to peer 1160c6e72055456992b5d8bf5f3b45c0. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.532058 18839 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.532081 16695 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.532474 21790 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.532831 16959 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.532927 17222 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.533205 21790 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.533448 22347 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.533854 16563 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.534531 16695 consensus_queue.cc:237] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:01:01.536892 19102 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.537045 17222 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.537163 17091 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240501 08:01:01.537276 16628 consensus_peers.cc:485] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d381bdb28e1f4374b81ab23f6882fe65. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.537297 16563 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.537667 22245 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.537878 22255 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.538101 22254 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.538190 18839 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.538489 22244 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.538837 22244 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.538894 21790 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.539093 17353 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.539345 22196 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.539489 22255 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.539628 22196 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.541498 16695 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.543395 17221 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.543578 16562 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.544633 16418 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 11 to 12, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.545156 18838 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.546131 16959 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.548702 18969 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.552202 22254 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.554855 17091 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.555490 16563 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.558974 16420 catalog_manager.cc:5484] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.559768 17353 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.565296 19102 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.565483 16959 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } }
I20240501 08:01:01.568063 17222 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.568779 18839 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.569226 18838 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:01.569763 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:01.569875 16695 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:01.571244 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:01.571292 16563 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
W20240501 08:01:01.571782 16892 consensus_peers.cc:485] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 -> Peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): Couldn't send request to peer 1160c6e72055456992b5d8bf5f3b45c0. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20240501 08:01:01.571892 16892 consensus_peers.cc:485] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5ecdf2c2cdb4487b0e6315f63e1b1b8. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.571985 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:01.574467 19102 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:01.575058 22101 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20240501 08:01:01.575232 16895 consensus_peers.cc:485] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 -> Peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): Couldn't send request to peer e282199dc2fa4977b83ff07b86cb0690. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.575611 17353 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:01.576265 17222 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:01.577769 22101 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:01.578122 22101 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:01.578426 22290 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.579278 17353 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.581947 16563 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.582892 16420 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: config changed from index 12 to 13, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:01:01.585232 19102 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.586283 16695 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.587242 18838 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.594236 17222 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:01.599536 16826 consensus_queue.cc:237] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:01:01.602465 16563 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.602465 18969 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.608558 22100 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.608870 16695 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.609607 17353 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.609625 17222 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.610656 17091 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.610831 22012 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 08:01:01.611214 16761 consensus_peers.cc:485] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 -> Peer 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138:37529): Couldn't send request to peer 5f8132e5a8014a6f913ee9ac3338600a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecd1ff8b89a84669bd8001ea731be7dc. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.611243 22012 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.611557 22012 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.611837 22012 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.612293 22100 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.616735 22393 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:01:01.617269 16563 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:01:01.620314 16695 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:01:01.621237 16413 catalog_manager.cc:5484] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 reported cstate change: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:01:01.621265 18969 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:01:01.629029 17222 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:01:01.630931 16695 consensus_queue.cc:237] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:01.633566 18839 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.633616 19102 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.633616 17089 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
W20240501 08:01:01.633857 16630 consensus_peers.cc:485] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 -> Peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): Couldn't send request to peer d5a55f6821d84c968a42f4a5c9ad07a7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.634111 17354 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.634088 22227 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.634454 22227 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.634797 22109 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.634963 22255 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.636282 17091 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:01:01.637701 16563 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.638120 22243 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.638080 22255 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
W20240501 08:01:01.639770 16628 consensus_peers.cc:485] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d381bdb28e1f4374b81ab23f6882fe65. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.641054 17091 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.641369 16415 catalog_manager.cc:5484] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 11 to 12, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:01:01.644979 17354 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.645471 18839 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.648763 16563 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.649864 17353 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } } }
I20240501 08:01:01.652137 19102 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.654693 17222 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:01.656981 22227 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:01.658696 17222 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.670894 17353 consensus_queue.cc:237] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:01:01.673207 18839 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.673789 17222 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.673944 19102 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.674103 22369 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.674546 22368 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.674794 16959 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.675071 22367 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.675580 22367 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.675817 16695 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.676990 16563 raft_consensus.cc:1261] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.677431 22367 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240501 08:01:01.677440 17288 consensus_peers.cc:485] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b -> Peer 873218458f2346348be2120e543990e2 (127.0.99.137:33839): Couldn't send request to peer 873218458f2346348be2120e543990e2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c6841a66987744f3bc06a4b2bdbc6df2. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.677641 22369 consensus_queue.cc:1035] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.680239 22369 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.682729 16563 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.682889 16695 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.684458 16959 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.686520 16416 catalog_manager.cc:5484] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:01:01.691679 18839 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.695624 17222 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.696571 16563 consensus_queue.cc:237] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:01.696780 19102 raft_consensus.cc:2932] T c6841a66987744f3bc06a4b2bdbc6df2 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } }
I20240501 08:01:01.699795 16826 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.699806 17353 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.700578 16695 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 08:01:01.700843 16498 consensus_peers.cc:485] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 -> Peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): Couldn't send request to peer d5a55f6821d84c968a42f4a5c9ad07a7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.701076 22319 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.701498 18969 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.703400 22333 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.704623 17091 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.705737 21988 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.706987 22319 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.707383 22319 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.708725 22333 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.709208 16695 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.710064 17222 raft_consensus.cc:1261] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 08:01:01.711601 21988 consensus_queue.cc:1035] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 08:01:01.712422 17353 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.713688 16415 catalog_manager.cc:5484] T 8bc2d601fe484082b947235237dbd42f P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 7 to 8, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.714468 17091 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.716118 18969 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.718437 17222 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.719915 16826 raft_consensus.cc:2932] T 8bc2d601fe484082b947235237dbd42f P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.733491 16695 consensus_queue.cc:237] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } }
I20240501 08:01:01.735841 17353 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.735865 19102 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.736342 18839 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.736440 22245 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.736681 16959 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.737589 17221 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.737661 16563 raft_consensus.cc:1261] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 08:01:01.737618 22227 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.737972 22410 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.738159 22227 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.738376 22243 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240501 08:01:01.739229 16631 consensus_peers.cc:485] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 -> Peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463): Couldn't send request to peer e282199dc2fa4977b83ff07b86cb0690. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.739573 22254 consensus_queue.cc:1035] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 08:01:01.742295 22254 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.744460 18839 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.745158 16563 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.746753 16417 catalog_manager.cc:5484] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 08:01:01.749276 17353 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.749672 16959 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.749850 17221 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.753542 19102 raft_consensus.cc:2932] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } }
I20240501 08:01:01.760397 17222 consensus_queue.cc:237] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:01.762684 17353 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:01:01.763087 18969 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:01:01.763377 16826 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:01:01.763676 22250 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:01.763798 17091 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
W20240501 08:01:01.763851 17158 consensus_peers.cc:485] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 -> Peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987): Couldn't send request to peer d5a55f6821d84c968a42f4a5c9ad07a7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.763975 21973 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:01.764426 21973 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:01.764581 16563 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
W20240501 08:01:01.764922 17157 consensus_peers.cc:485] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 -> Peer 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136:34371): Couldn't send request to peer 5aaf071b45ab469cb23321d37fad37f8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d27d6c2cc194773a648909806f4416b. This is attempt 1: this message will repeat every 5th retry.
I20240501 08:01:01.765324 16695 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:01:01.766062 22250 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:01.766391 22250 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:01.767323 21973 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:01.769574 22251 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.770315 16563 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.772073 17353 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.772135 16826 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.773478 16419 catalog_manager.cc:5484] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 8 to 9, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.776840 16695 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.777946 17091 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.778321 18969 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:01.830296 22520 ts_tablet_manager.cc:902] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Initiating tablet copy from peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:01:01.831178 22520 tablet_copy_client.cc:308] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:43741
I20240501 08:01:01.831864 16583 tablet_copy_service.cc:140] P eca6f08d57d1453784f726199474a671: Received BeginTabletCopySession request for tablet 5ae979c5adf44f3e8590ef06c71bf904 from peer d5a55f6821d84c968a42f4a5c9ad07a7 ({username='slave'} at 127.0.99.132:47965)
I20240501 08:01:01.832038 16583 tablet_copy_service.cc:161] P eca6f08d57d1453784f726199474a671: Beginning new tablet copy session on tablet 5ae979c5adf44f3e8590ef06c71bf904 from peer d5a55f6821d84c968a42f4a5c9ad07a7 at {username='slave'} at 127.0.99.132:47965: session id = d5a55f6821d84c968a42f4a5c9ad07a7-5ae979c5adf44f3e8590ef06c71bf904
I20240501 08:01:01.833138 16583 tablet_copy_source_session.cc:205] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:01.834409 22520 ts_tablet_manager.cc:1875] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 08:01:01.837033 22520 ts_tablet_manager.cc:1888] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 08:01:01.837263 22520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ae979c5adf44f3e8590ef06c71bf904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:01.838369 22520 tablet_copy_client.cc:783] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:01.838544 22520 tablet_copy_client.cc:647] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:01.840947 22520 tablet_copy_client.cc:523] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:01.842659 22520 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:01:01.866904 22520 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:01.867259 22520 tablet_bootstrap.cc:492] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap complete.
I20240501 08:01:01.867455 22520 ts_tablet_manager.cc:1367] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.025s user 0.023s sys 0.000s
I20240501 08:01:01.867954 22520 raft_consensus.cc:348] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:01.868098 22520 raft_consensus.cc:729] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: LEARNER
I20240501 08:01:01.868191 22520 consensus_queue.cc:260] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:01.868512 22520 ts_tablet_manager.cc:1398] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.004s
I20240501 08:01:01.869235 16583 tablet_copy_service.cc:342] P eca6f08d57d1453784f726199474a671: Request end of tablet copy session d5a55f6821d84c968a42f4a5c9ad07a7-5ae979c5adf44f3e8590ef06c71bf904 received from {username='slave'} at 127.0.99.132:47965
I20240501 08:01:01.869343 16583 tablet_copy_service.cc:433] P eca6f08d57d1453784f726199474a671: ending tablet copy session d5a55f6821d84c968a42f4a5c9ad07a7-5ae979c5adf44f3e8590ef06c71bf904 on tablet 5ae979c5adf44f3e8590ef06c71bf904 with peer d5a55f6821d84c968a42f4a5c9ad07a7
I20240501 08:01:01.924759 22531 ts_tablet_manager.cc:902] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Initiating tablet copy from peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463)
I20240501 08:01:01.925638 22531 tablet_copy_client.cc:308] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:41463
I20240501 08:01:01.926364 17111 tablet_copy_service.cc:140] P e282199dc2fa4977b83ff07b86cb0690: Received BeginTabletCopySession request for tablet 452b5685e6ca4793a6628d700599bdbb from peer eca6f08d57d1453784f726199474a671 ({username='slave'} at 127.0.99.129:55393)
I20240501 08:01:01.926504 17111 tablet_copy_service.cc:161] P e282199dc2fa4977b83ff07b86cb0690: Beginning new tablet copy session on tablet 452b5685e6ca4793a6628d700599bdbb from peer eca6f08d57d1453784f726199474a671 at {username='slave'} at 127.0.99.129:55393: session id = eca6f08d57d1453784f726199474a671-452b5685e6ca4793a6628d700599bdbb
I20240501 08:01:01.927383 17111 tablet_copy_source_session.cc:205] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:01.928596 22531 ts_tablet_manager.cc:1875] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 08:01:01.930797 22531 ts_tablet_manager.cc:1888] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240501 08:01:01.931028 22531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 452b5685e6ca4793a6628d700599bdbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:01.933483 22531 tablet_copy_client.cc:783] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:01.933717 22531 tablet_copy_client.cc:647] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:01.935824 22531 tablet_copy_client.cc:523] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:01.938385 22531 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:01:01.939512 22520 ts_tablet_manager.cc:902] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Initiating tablet copy from peer 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219)
I20240501 08:01:01.940069 22520 tablet_copy_client.cc:308] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:45219
I20240501 08:01:01.940644 16715 tablet_copy_service.cc:140] P 64196487691e4623b690b2869aad57f7: Received BeginTabletCopySession request for tablet 6fd34b65ff134bf485af45d1eccd62f9 from peer d5a55f6821d84c968a42f4a5c9ad07a7 ({username='slave'} at 127.0.99.132:58205)
I20240501 08:01:01.940775 16715 tablet_copy_service.cc:161] P 64196487691e4623b690b2869aad57f7: Beginning new tablet copy session on tablet 6fd34b65ff134bf485af45d1eccd62f9 from peer d5a55f6821d84c968a42f4a5c9ad07a7 at {username='slave'} at 127.0.99.132:58205: session id = d5a55f6821d84c968a42f4a5c9ad07a7-6fd34b65ff134bf485af45d1eccd62f9
I20240501 08:01:01.941812 16715 tablet_copy_source_session.cc:205] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:01.942785 22520 ts_tablet_manager.cc:1875] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 08:01:01.945585 22520 ts_tablet_manager.cc:1888] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 08:01:01.945791 22520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd34b65ff134bf485af45d1eccd62f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:01.946970 22520 tablet_copy_client.cc:783] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:01.947134 22520 tablet_copy_client.cc:647] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:01.949020 22520 tablet_copy_client.cc:523] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:01.950682 22520 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:01:01.970264 22531 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:01.970687 22531 tablet_bootstrap.cc:492] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Bootstrap complete.
I20240501 08:01:01.970948 22531 ts_tablet_manager.cc:1367] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.033s user 0.025s sys 0.005s
I20240501 08:01:01.971501 22531 raft_consensus.cc:348] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } }
I20240501 08:01:01.971670 22531 raft_consensus.cc:729] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: LEARNER
I20240501 08:01:01.971777 22531 consensus_queue.cc:260] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } }
I20240501 08:01:01.972178 22531 ts_tablet_manager.cc:1398] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 08:01:01.973405 17111 tablet_copy_service.cc:342] P e282199dc2fa4977b83ff07b86cb0690: Request end of tablet copy session eca6f08d57d1453784f726199474a671-452b5685e6ca4793a6628d700599bdbb received from {username='slave'} at 127.0.99.129:55393
I20240501 08:01:01.973547 17111 tablet_copy_service.cc:433] P e282199dc2fa4977b83ff07b86cb0690: ending tablet copy session eca6f08d57d1453784f726199474a671-452b5685e6ca4793a6628d700599bdbb on tablet 452b5685e6ca4793a6628d700599bdbb with peer eca6f08d57d1453784f726199474a671
I20240501 08:01:01.982394 22520 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:01.982733 22520 tablet_bootstrap.cc:492] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap complete.
I20240501 08:01:01.982935 22520 ts_tablet_manager.cc:1367] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.032s user 0.020s sys 0.007s
I20240501 08:01:01.983435 22520 raft_consensus.cc:348] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:01.983579 22520 raft_consensus.cc:729] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: LEARNER
I20240501 08:01:01.983665 22520 consensus_queue.cc:260] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 2.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:01.983978 22520 ts_tablet_manager.cc:1398] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 08:01:01.984625 16715 tablet_copy_service.cc:342] P 64196487691e4623b690b2869aad57f7: Request end of tablet copy session d5a55f6821d84c968a42f4a5c9ad07a7-6fd34b65ff134bf485af45d1eccd62f9 received from {username='slave'} at 127.0.99.132:58205
I20240501 08:01:01.984726 16715 tablet_copy_service.cc:433] P 64196487691e4623b690b2869aad57f7: ending tablet copy session d5a55f6821d84c968a42f4a5c9ad07a7-6fd34b65ff134bf485af45d1eccd62f9 on tablet 6fd34b65ff134bf485af45d1eccd62f9 with peer d5a55f6821d84c968a42f4a5c9ad07a7
I20240501 08:01:02.004678 22572 ts_tablet_manager.cc:902] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Initiating tablet copy from peer af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:01:02.005988 22576 ts_tablet_manager.cc:902] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Initiating tablet copy from peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987)
I20240501 08:01:02.006068 22572 tablet_copy_client.cc:308] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:39065
I20240501 08:01:02.006718 17375 tablet_copy_service.cc:140] P af70005d0c6c48bc9919cae6346f221b: Received BeginTabletCopySession request for tablet fa0d919844a349549b1ad6ffbe440837 from peer e282199dc2fa4977b83ff07b86cb0690 ({username='slave'} at 127.0.99.133:51847)
I20240501 08:01:02.006820 22576 tablet_copy_client.cc:308] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:41987
I20240501 08:01:02.006850 17375 tablet_copy_service.cc:161] P af70005d0c6c48bc9919cae6346f221b: Beginning new tablet copy session on tablet fa0d919844a349549b1ad6ffbe440837 from peer e282199dc2fa4977b83ff07b86cb0690 at {username='slave'} at 127.0.99.133:51847: session id = e282199dc2fa4977b83ff07b86cb0690-fa0d919844a349549b1ad6ffbe440837
I20240501 08:01:02.007457 16979 tablet_copy_service.cc:140] P d5a55f6821d84c968a42f4a5c9ad07a7: Received BeginTabletCopySession request for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 from peer 1160c6e72055456992b5d8bf5f3b45c0 ({username='slave'} at 127.0.99.131:35131)
I20240501 08:01:02.007579 16979 tablet_copy_service.cc:161] P d5a55f6821d84c968a42f4a5c9ad07a7: Beginning new tablet copy session on tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 from peer 1160c6e72055456992b5d8bf5f3b45c0 at {username='slave'} at 127.0.99.131:35131: session id = 1160c6e72055456992b5d8bf5f3b45c0-d5ecdf2c2cdb4487b0e6315f63e1b1b8
I20240501 08:01:02.007896 17375 tablet_copy_source_session.cc:205] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.008491 16979 tablet_copy_source_session.cc:205] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.009594 22576 ts_tablet_manager.cc:1875] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 08:01:02.009595 22572 ts_tablet_manager.cc:1875] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 08:01:02.012101 22572 ts_tablet_manager.cc:1888] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 08:01:02.012347 22572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0d919844a349549b1ad6ffbe440837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.012559 22576 ts_tablet_manager.cc:1888] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240501 08:01:02.012753 22576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.013844 22572 tablet_copy_client.cc:783] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.014032 22572 tablet_copy_client.cc:647] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.014091 22576 tablet_copy_client.cc:783] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.014288 22576 tablet_copy_client.cc:647] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.015866 22572 tablet_copy_client.cc:523] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.016129 22576 tablet_copy_client.cc:523] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.018168 22572 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:01:02.018232 22576 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:01:02.021080 22592 ts_tablet_manager.cc:902] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a: Initiating tablet copy from peer 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779)
I20240501 08:01:02.022861 22592 tablet_copy_client.cc:308] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:43779
I20240501 08:01:02.023455 16847 tablet_copy_service.cc:140] P 1160c6e72055456992b5d8bf5f3b45c0: Received BeginTabletCopySession request for tablet ecd1ff8b89a84669bd8001ea731be7dc from peer 5f8132e5a8014a6f913ee9ac3338600a ({username='slave'} at 127.0.99.138:45789)
I20240501 08:01:02.023566 16847 tablet_copy_service.cc:161] P 1160c6e72055456992b5d8bf5f3b45c0: Beginning new tablet copy session on tablet ecd1ff8b89a84669bd8001ea731be7dc from peer 5f8132e5a8014a6f913ee9ac3338600a at {username='slave'} at 127.0.99.138:45789: session id = 5f8132e5a8014a6f913ee9ac3338600a-ecd1ff8b89a84669bd8001ea731be7dc
I20240501 08:01:02.024451 16847 tablet_copy_source_session.cc:205] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.025398 22592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd1ff8b89a84669bd8001ea731be7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.026540 22597 ts_tablet_manager.cc:902] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987)
I20240501 08:01:02.027108 22597 tablet_copy_client.cc:308] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:41987
I20240501 08:01:02.028056 22592 tablet_copy_client.cc:783] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.028231 22592 tablet_copy_client.cc:647] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.029748 22592 tablet_copy_client.cc:523] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.031569 22592 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap starting.
I20240501 08:01:02.033171 16979 tablet_copy_service.cc:140] P d5a55f6821d84c968a42f4a5c9ad07a7: Received BeginTabletCopySession request for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:33271)
I20240501 08:01:02.033291 16979 tablet_copy_service.cc:161] P d5a55f6821d84c968a42f4a5c9ad07a7: Beginning new tablet copy session on tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:33271: session id = 873218458f2346348be2120e543990e2-d5ecdf2c2cdb4487b0e6315f63e1b1b8
I20240501 08:01:02.034258 16979 tablet_copy_source_session.cc:205] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.048781 22597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.049088 22531 ts_tablet_manager.cc:902] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Initiating tablet copy from peer e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133:41463)
I20240501 08:01:02.049623 22531 tablet_copy_client.cc:308] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:41463
I20240501 08:01:02.050400 17111 tablet_copy_service.cc:140] P e282199dc2fa4977b83ff07b86cb0690: Received BeginTabletCopySession request for tablet 5e0b477de25646eda4f9f0e4921ec118 from peer eca6f08d57d1453784f726199474a671 ({username='slave'} at 127.0.99.129:55393)
I20240501 08:01:02.050520 17111 tablet_copy_service.cc:161] P e282199dc2fa4977b83ff07b86cb0690: Beginning new tablet copy session on tablet 5e0b477de25646eda4f9f0e4921ec118 from peer eca6f08d57d1453784f726199474a671 at {username='slave'} at 127.0.99.129:55393: session id = eca6f08d57d1453784f726199474a671-5e0b477de25646eda4f9f0e4921ec118
I20240501 08:01:02.051587 17111 tablet_copy_source_session.cc:205] T 5e0b477de25646eda4f9f0e4921ec118 P e282199dc2fa4977b83ff07b86cb0690: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.052754 22597 tablet_copy_client.cc:783] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.052946 22597 tablet_copy_client.cc:647] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.053041 22531 ts_tablet_manager.cc:1875] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 08:01:02.054188 22597 tablet_copy_client.cc:523] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.055495 22531 ts_tablet_manager.cc:1888] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240501 08:01:02.055685 22531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e0b477de25646eda4f9f0e4921ec118. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.055713 22597 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:01:02.059571 22576 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.060010 22576 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap complete.
I20240501 08:01:02.060226 22576 ts_tablet_manager.cc:1367] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.042s user 0.033s sys 0.000s
I20240501 08:01:02.060739 22576 raft_consensus.cc:348] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } }
I20240501 08:01:02.060914 22576 raft_consensus.cc:729] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: LEARNER
I20240501 08:01:02.060881 22531 tablet_copy_client.cc:783] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.061028 22576 consensus_queue.cc:260] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 1.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } }
I20240501 08:01:02.061336 22531 tablet_copy_client.cc:647] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.062052 22576 ts_tablet_manager.cc:1398] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:01:02.062602 16979 tablet_copy_service.cc:342] P d5a55f6821d84c968a42f4a5c9ad07a7: Request end of tablet copy session 1160c6e72055456992b5d8bf5f3b45c0-d5ecdf2c2cdb4487b0e6315f63e1b1b8 received from {username='slave'} at 127.0.99.131:35131
I20240501 08:01:02.062707 16979 tablet_copy_service.cc:433] P d5a55f6821d84c968a42f4a5c9ad07a7: ending tablet copy session 1160c6e72055456992b5d8bf5f3b45c0-d5ecdf2c2cdb4487b0e6315f63e1b1b8 on tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with peer 1160c6e72055456992b5d8bf5f3b45c0
I20240501 08:01:02.062940 22531 tablet_copy_client.cc:523] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.064571 22531 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Bootstrap starting.
I20240501 08:01:02.067975 22572 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.068331 22572 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap complete.
I20240501 08:01:02.068534 22572 ts_tablet_manager.cc:1367] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.050s user 0.032s sys 0.001s
I20240501 08:01:02.069006 22572 raft_consensus.cc:348] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } }
I20240501 08:01:02.069178 22572 raft_consensus.cc:729] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: LEARNER
I20240501 08:01:02.069286 22572 consensus_queue.cc:260] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } }
I20240501 08:01:02.069504 22576 ts_tablet_manager.cc:902] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Initiating tablet copy from peer af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:01:02.069700 22572 ts_tablet_manager.cc:1398] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:01:02.069970 22576 tablet_copy_client.cc:308] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:39065
I20240501 08:01:02.070441 17375 tablet_copy_service.cc:342] P af70005d0c6c48bc9919cae6346f221b: Request end of tablet copy session e282199dc2fa4977b83ff07b86cb0690-fa0d919844a349549b1ad6ffbe440837 received from {username='slave'} at 127.0.99.133:51847
I20240501 08:01:02.070523 17375 tablet_copy_service.cc:433] P af70005d0c6c48bc9919cae6346f221b: ending tablet copy session e282199dc2fa4977b83ff07b86cb0690-fa0d919844a349549b1ad6ffbe440837 on tablet fa0d919844a349549b1ad6ffbe440837 with peer e282199dc2fa4977b83ff07b86cb0690
I20240501 08:01:02.071766 17374 tablet_copy_service.cc:140] P af70005d0c6c48bc9919cae6346f221b: Received BeginTabletCopySession request for tablet fa0d919844a349549b1ad6ffbe440837 from peer 1160c6e72055456992b5d8bf5f3b45c0 ({username='slave'} at 127.0.99.131:59715)
I20240501 08:01:02.071872 17374 tablet_copy_service.cc:161] P af70005d0c6c48bc9919cae6346f221b: Beginning new tablet copy session on tablet fa0d919844a349549b1ad6ffbe440837 from peer 1160c6e72055456992b5d8bf5f3b45c0 at {username='slave'} at 127.0.99.131:59715: session id = 1160c6e72055456992b5d8bf5f3b45c0-fa0d919844a349549b1ad6ffbe440837
I20240501 08:01:02.072886 17374 tablet_copy_source_session.cc:205] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.075001 22572 ts_tablet_manager.cc:902] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Initiating tablet copy from peer d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132:41987)
I20240501 08:01:02.075749 22572 tablet_copy_client.cc:308] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:41987
I20240501 08:01:02.076633 16979 tablet_copy_service.cc:140] P d5a55f6821d84c968a42f4a5c9ad07a7: Received BeginTabletCopySession request for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 from peer e282199dc2fa4977b83ff07b86cb0690 ({username='slave'} at 127.0.99.133:49265)
I20240501 08:01:02.076762 16979 tablet_copy_service.cc:161] P d5a55f6821d84c968a42f4a5c9ad07a7: Beginning new tablet copy session on tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 from peer e282199dc2fa4977b83ff07b86cb0690 at {username='slave'} at 127.0.99.133:49265: session id = e282199dc2fa4977b83ff07b86cb0690-d5ecdf2c2cdb4487b0e6315f63e1b1b8
I20240501 08:01:02.077039 22576 ts_tablet_manager.cc:1875] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 08:01:02.077708 16979 tablet_copy_source_session.cc:205] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.078119 22621 ts_tablet_manager.cc:902] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065)
I20240501 08:01:02.078625 22572 ts_tablet_manager.cc:1875] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 08:01:02.078702 22621 tablet_copy_client.cc:308] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:39065
I20240501 08:01:02.079288 17374 tablet_copy_service.cc:140] P af70005d0c6c48bc9919cae6346f221b: Received BeginTabletCopySession request for tablet c6841a66987744f3bc06a4b2bdbc6df2 from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:53219)
I20240501 08:01:02.079414 17374 tablet_copy_service.cc:161] P af70005d0c6c48bc9919cae6346f221b: Beginning new tablet copy session on tablet c6841a66987744f3bc06a4b2bdbc6df2 from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:53219: session id = 873218458f2346348be2120e543990e2-c6841a66987744f3bc06a4b2bdbc6df2
I20240501 08:01:02.079681 22576 ts_tablet_manager.cc:1888] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240501 08:01:02.080616 17374 tablet_copy_source_session.cc:205] T c6841a66987744f3bc06a4b2bdbc6df2 P af70005d0c6c48bc9919cae6346f221b: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.080672 22576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa0d919844a349549b1ad6ffbe440837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.081692 22621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6841a66987744f3bc06a4b2bdbc6df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.082232 22576 tablet_copy_client.cc:783] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.082669 22576 tablet_copy_client.cc:647] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.085150 22572 ts_tablet_manager.cc:1888] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 08:01:02.085373 22572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.087229 22572 tablet_copy_client.cc:783] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.087442 22572 tablet_copy_client.cc:647] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.089174 22572 tablet_copy_client.cc:523] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.091562 22572 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:01:02.097139 22621 tablet_copy_client.cc:783] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.097368 22621 tablet_copy_client.cc:647] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.099314 22626 ts_tablet_manager.cc:902] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2: Initiating tablet copy from peer 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219)
I20240501 08:01:02.099985 22626 tablet_copy_client.cc:308] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:45219
I20240501 08:01:02.101143 22531 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.101691 22531 tablet_bootstrap.cc:492] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Bootstrap complete.
I20240501 08:01:02.101925 22531 ts_tablet_manager.cc:1367] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Time spent bootstrapping tablet: real 0.037s user 0.028s sys 0.003s
I20240501 08:01:02.103986 22621 tablet_copy_client.cc:523] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.105803 22621 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:01:02.110654 22576 tablet_copy_client.cc:523] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.112480 22576 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap starting.
I20240501 08:01:02.118225 22531 raft_consensus.cc:348] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } }
I20240501 08:01:02.118553 22531 raft_consensus.cc:729] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eca6f08d57d1453784f726199474a671, State: Initialized, Role: LEARNER
I20240501 08:01:02.118732 22531 consensus_queue.cc:260] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: NON_VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: true } }
I20240501 08:01:02.119892 22531 ts_tablet_manager.cc:1398] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671: Time spent starting tablet: real 0.018s user 0.001s sys 0.000s
I20240501 08:01:02.120527 17111 tablet_copy_service.cc:342] P e282199dc2fa4977b83ff07b86cb0690: Request end of tablet copy session eca6f08d57d1453784f726199474a671-5e0b477de25646eda4f9f0e4921ec118 received from {username='slave'} at 127.0.99.129:55393
I20240501 08:01:02.120622 17111 tablet_copy_service.cc:433] P e282199dc2fa4977b83ff07b86cb0690: ending tablet copy session eca6f08d57d1453784f726199474a671-5e0b477de25646eda4f9f0e4921ec118 on tablet 5e0b477de25646eda4f9f0e4921ec118 with peer eca6f08d57d1453784f726199474a671
I20240501 08:01:02.139894 22572 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.140139 22592 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.140445 22592 tablet_bootstrap.cc:492] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a: Bootstrap complete.
I20240501 08:01:02.140484 22572 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap complete.
I20240501 08:01:02.140667 22592 ts_tablet_manager.cc:1367] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a: Time spent bootstrapping tablet: real 0.109s user 0.031s sys 0.000s
I20240501 08:01:02.140667 22572 ts_tablet_manager.cc:1367] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.049s user 0.025s sys 0.004s
I20240501 08:01:02.141165 22592 raft_consensus.cc:348] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:01:02.141165 22572 raft_consensus.cc:348] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } }
I20240501 08:01:02.141319 22572 raft_consensus.cc:729] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: LEARNER
I20240501 08:01:02.141319 22592 raft_consensus.cc:729] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f8132e5a8014a6f913ee9ac3338600a, State: Initialized, Role: LEARNER
I20240501 08:01:02.141434 22592 consensus_queue.cc:260] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: NON_VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: true } }
I20240501 08:01:02.141434 22572 consensus_queue.cc:260] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 1.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } }
I20240501 08:01:02.142107 22572 ts_tablet_manager.cc:1398] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 08:01:02.143000 16979 tablet_copy_service.cc:342] P d5a55f6821d84c968a42f4a5c9ad07a7: Request end of tablet copy session e282199dc2fa4977b83ff07b86cb0690-d5ecdf2c2cdb4487b0e6315f63e1b1b8 received from {username='slave'} at 127.0.99.133:49265
I20240501 08:01:02.143097 16979 tablet_copy_service.cc:433] P d5a55f6821d84c968a42f4a5c9ad07a7: ending tablet copy session e282199dc2fa4977b83ff07b86cb0690-d5ecdf2c2cdb4487b0e6315f63e1b1b8 on tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with peer e282199dc2fa4977b83ff07b86cb0690
I20240501 08:01:02.143817 16715 tablet_copy_service.cc:140] P 64196487691e4623b690b2869aad57f7: Received BeginTabletCopySession request for tablet d381bdb28e1f4374b81ab23f6882fe65 from peer 873218458f2346348be2120e543990e2 ({username='slave'} at 127.0.99.137:44661)
I20240501 08:01:02.143955 16715 tablet_copy_service.cc:161] P 64196487691e4623b690b2869aad57f7: Beginning new tablet copy session on tablet d381bdb28e1f4374b81ab23f6882fe65 from peer 873218458f2346348be2120e543990e2 at {username='slave'} at 127.0.99.137:44661: session id = 873218458f2346348be2120e543990e2-d381bdb28e1f4374b81ab23f6882fe65
I20240501 08:01:02.144953 16715 tablet_copy_source_session.cc:205] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.146004 22626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d381bdb28e1f4374b81ab23f6882fe65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.148226 22626 tablet_copy_client.cc:783] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.148403 22626 tablet_copy_client.cc:647] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.150302 22626 tablet_copy_client.cc:523] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.151594 22626 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2: Bootstrap starting.
I20240501 08:01:02.152654 22592 ts_tablet_manager.cc:1398] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20240501 08:01:02.153214 16847 tablet_copy_service.cc:342] P 1160c6e72055456992b5d8bf5f3b45c0: Request end of tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-ecd1ff8b89a84669bd8001ea731be7dc received from {username='slave'} at 127.0.99.138:45789
I20240501 08:01:02.153301 16847 tablet_copy_service.cc:433] P 1160c6e72055456992b5d8bf5f3b45c0: ending tablet copy session 5f8132e5a8014a6f913ee9ac3338600a-ecd1ff8b89a84669bd8001ea731be7dc on tablet ecd1ff8b89a84669bd8001ea731be7dc with peer 5f8132e5a8014a6f913ee9ac3338600a
I20240501 08:01:02.187376 22576 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.188632 22520 ts_tablet_manager.cc:902] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Initiating tablet copy from peer eca6f08d57d1453784f726199474a671 (127.0.99.129:43741)
I20240501 08:01:02.189158 22520 tablet_copy_client.cc:308] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:43741
I20240501 08:01:02.189667 16583 tablet_copy_service.cc:140] P eca6f08d57d1453784f726199474a671: Received BeginTabletCopySession request for tablet 8bc2d601fe484082b947235237dbd42f from peer d5a55f6821d84c968a42f4a5c9ad07a7 ({username='slave'} at 127.0.99.132:47965)
I20240501 08:01:02.189788 16583 tablet_copy_service.cc:161] P eca6f08d57d1453784f726199474a671: Beginning new tablet copy session on tablet 8bc2d601fe484082b947235237dbd42f from peer d5a55f6821d84c968a42f4a5c9ad07a7 at {username='slave'} at 127.0.99.132:47965: session id = d5a55f6821d84c968a42f4a5c9ad07a7-8bc2d601fe484082b947235237dbd42f
I20240501 08:01:02.190438 22597 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.190742 22597 tablet_bootstrap.cc:492] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:01:02.190810 16583 tablet_copy_source_session.cc:205] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.191005 22597 ts_tablet_manager.cc:1367] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.135s user 0.028s sys 0.004s
I20240501 08:01:02.191557 22597 raft_consensus.cc:348] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } }
I20240501 08:01:02.191730 22597 raft_consensus.cc:729] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:01:02.191831 22520 ts_tablet_manager.cc:1875] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 08:01:02.191838 22597 consensus_queue.cc:260] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 1.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } }
I20240501 08:01:02.192339 22597 ts_tablet_manager.cc:1398] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:01:02.195084 16979 tablet_copy_service.cc:342] P d5a55f6821d84c968a42f4a5c9ad07a7: Request end of tablet copy session 873218458f2346348be2120e543990e2-d5ecdf2c2cdb4487b0e6315f63e1b1b8 received from {username='slave'} at 127.0.99.137:33271
I20240501 08:01:02.195183 16979 tablet_copy_service.cc:433] P d5a55f6821d84c968a42f4a5c9ad07a7: ending tablet copy session 873218458f2346348be2120e543990e2-d5ecdf2c2cdb4487b0e6315f63e1b1b8 on tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with peer 873218458f2346348be2120e543990e2
I20240501 08:01:02.196420 22626 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.197449 22626 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:01:02.197710 22626 ts_tablet_manager.cc:1367] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.046s user 0.027s sys 0.005s
I20240501 08:01:02.198379 22520 ts_tablet_manager.cc:1888] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 08:01:02.198302 22626 raft_consensus.cc:348] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:02.198593 22520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bc2d601fe484082b947235237dbd42f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.198711 22626 raft_consensus.cc:729] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:01:02.198884 22626 consensus_queue.cc:260] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:02.199470 22626 ts_tablet_manager.cc:1398] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:01:02.200349 16715 tablet_copy_service.cc:342] P 64196487691e4623b690b2869aad57f7: Request end of tablet copy session 873218458f2346348be2120e543990e2-d381bdb28e1f4374b81ab23f6882fe65 received from {username='slave'} at 127.0.99.137:44661
I20240501 08:01:02.200443 16715 tablet_copy_service.cc:433] P 64196487691e4623b690b2869aad57f7: ending tablet copy session 873218458f2346348be2120e543990e2-d381bdb28e1f4374b81ab23f6882fe65 on tablet d381bdb28e1f4374b81ab23f6882fe65 with peer 873218458f2346348be2120e543990e2
I20240501 08:01:02.201120 22520 tablet_copy_client.cc:783] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.201269 22676 ts_tablet_manager.cc:902] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8: Initiating tablet copy from peer 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813)
I20240501 08:01:02.201292 22520 tablet_copy_client.cc:647] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.201462 22576 tablet_bootstrap.cc:492] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Bootstrap complete.
I20240501 08:01:02.201793 22576 ts_tablet_manager.cc:1367] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent bootstrapping tablet: real 0.089s user 0.028s sys 0.004s
I20240501 08:01:02.201833 22676 tablet_copy_client.cc:308] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:46813
I20240501 08:01:02.202368 17243 tablet_copy_service.cc:140] P 3d06b1f1890443da898fde2eb82daad3: Received BeginTabletCopySession request for tablet 0d27d6c2cc194773a648909806f4416b from peer 5aaf071b45ab469cb23321d37fad37f8 ({username='slave'} at 127.0.99.136:44473)
I20240501 08:01:02.202476 17243 tablet_copy_service.cc:161] P 3d06b1f1890443da898fde2eb82daad3: Beginning new tablet copy session on tablet 0d27d6c2cc194773a648909806f4416b from peer 5aaf071b45ab469cb23321d37fad37f8 at {username='slave'} at 127.0.99.136:44473: session id = 5aaf071b45ab469cb23321d37fad37f8-0d27d6c2cc194773a648909806f4416b
I20240501 08:01:02.203034 22576 raft_consensus.cc:348] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } }
I20240501 08:01:02.203183 22576 raft_consensus.cc:729] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1160c6e72055456992b5d8bf5f3b45c0, State: Initialized, Role: LEARNER
I20240501 08:01:02.203294 22576 consensus_queue.cc:260] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } }
I20240501 08:01:02.203456 17243 tablet_copy_source_session.cc:205] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.205272 22676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d27d6c2cc194773a648909806f4416b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.205272 22576 ts_tablet_manager.cc:1398] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 08:01:02.206099 22621 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.206403 22621 tablet_bootstrap.cc:492] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2: Bootstrap complete.
I20240501 08:01:02.206629 22621 ts_tablet_manager.cc:1367] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2: Time spent bootstrapping tablet: real 0.101s user 0.029s sys 0.002s
I20240501 08:01:02.206732 17374 tablet_copy_service.cc:342] P af70005d0c6c48bc9919cae6346f221b: Request end of tablet copy session 1160c6e72055456992b5d8bf5f3b45c0-fa0d919844a349549b1ad6ffbe440837 received from {username='slave'} at 127.0.99.131:59715
I20240501 08:01:02.206945 17374 tablet_copy_service.cc:433] P af70005d0c6c48bc9919cae6346f221b: ending tablet copy session 1160c6e72055456992b5d8bf5f3b45c0-fa0d919844a349549b1ad6ffbe440837 on tablet fa0d919844a349549b1ad6ffbe440837 with peer 1160c6e72055456992b5d8bf5f3b45c0
I20240501 08:01:02.207167 22621 raft_consensus.cc:348] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:01:02.207330 22621 raft_consensus.cc:729] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 873218458f2346348be2120e543990e2, State: Initialized, Role: LEARNER
I20240501 08:01:02.207418 22621 consensus_queue.cc:260] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } }
I20240501 08:01:02.207700 22621 ts_tablet_manager.cc:1398] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:01:02.210107 22676 tablet_copy_client.cc:783] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.210386 22676 tablet_copy_client.cc:647] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.210634 17374 tablet_copy_service.cc:342] P af70005d0c6c48bc9919cae6346f221b: Request end of tablet copy session 873218458f2346348be2120e543990e2-c6841a66987744f3bc06a4b2bdbc6df2 received from {username='slave'} at 127.0.99.137:53219
I20240501 08:01:02.210738 17374 tablet_copy_service.cc:433] P af70005d0c6c48bc9919cae6346f221b: ending tablet copy session 873218458f2346348be2120e543990e2-c6841a66987744f3bc06a4b2bdbc6df2 on tablet c6841a66987744f3bc06a4b2bdbc6df2 with peer 873218458f2346348be2120e543990e2
I20240501 08:01:02.211423 22520 tablet_copy_client.cc:523] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.213205 22520 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:01:02.214820 22676 tablet_copy_client.cc:523] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.216439 22676 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap starting.
I20240501 08:01:02.221966 22691 ts_tablet_manager.cc:902] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Initiating tablet copy from peer 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219)
I20240501 08:01:02.224951 22691 tablet_copy_client.cc:308] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:45219
I20240501 08:01:02.226245 16715 tablet_copy_service.cc:140] P 64196487691e4623b690b2869aad57f7: Received BeginTabletCopySession request for tablet d381bdb28e1f4374b81ab23f6882fe65 from peer d5a55f6821d84c968a42f4a5c9ad07a7 ({username='slave'} at 127.0.99.132:58205)
I20240501 08:01:02.226379 16715 tablet_copy_service.cc:161] P 64196487691e4623b690b2869aad57f7: Beginning new tablet copy session on tablet d381bdb28e1f4374b81ab23f6882fe65 from peer d5a55f6821d84c968a42f4a5c9ad07a7 at {username='slave'} at 127.0.99.132:58205: session id = d5a55f6821d84c968a42f4a5c9ad07a7-d381bdb28e1f4374b81ab23f6882fe65
I20240501 08:01:02.227319 16715 tablet_copy_source_session.cc:205] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.228181 22691 ts_tablet_manager.cc:1875] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 08:01:02.230562 22691 ts_tablet_manager.cc:1888] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 08:01:02.230753 22691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d381bdb28e1f4374b81ab23f6882fe65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.234856 22691 tablet_copy_client.cc:783] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.235563 22691 tablet_copy_client.cc:647] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.237696 22691 tablet_copy_client.cc:523] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.239414 22691 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:01:02.249450 22520 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.249761 22520 tablet_bootstrap.cc:492] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap complete.
I20240501 08:01:02.249953 22520 ts_tablet_manager.cc:1367] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.037s user 0.027s sys 0.000s
I20240501 08:01:02.250435 22520 raft_consensus.cc:348] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:02.250579 22520 raft_consensus.cc:729] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: LEARNER
I20240501 08:01:02.250705 22520 consensus_queue.cc:260] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:02.251015 22520 ts_tablet_manager.cc:1398] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 08:01:02.251551 16583 tablet_copy_service.cc:342] P eca6f08d57d1453784f726199474a671: Request end of tablet copy session d5a55f6821d84c968a42f4a5c9ad07a7-8bc2d601fe484082b947235237dbd42f received from {username='slave'} at 127.0.99.132:47965
I20240501 08:01:02.251627 16583 tablet_copy_service.cc:433] P eca6f08d57d1453784f726199474a671: ending tablet copy session d5a55f6821d84c968a42f4a5c9ad07a7-8bc2d601fe484082b947235237dbd42f on tablet 8bc2d601fe484082b947235237dbd42f with peer d5a55f6821d84c968a42f4a5c9ad07a7
I20240501 08:01:02.263065 22676 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.263422 22676 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8: Bootstrap complete.
I20240501 08:01:02.263615 22676 ts_tablet_manager.cc:1367] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8: Time spent bootstrapping tablet: real 0.047s user 0.018s sys 0.006s
I20240501 08:01:02.264129 22676 raft_consensus.cc:348] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:02.265401 22676 raft_consensus.cc:729] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5aaf071b45ab469cb23321d37fad37f8, State: Initialized, Role: LEARNER
I20240501 08:01:02.265530 22676 consensus_queue.cc:260] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:02.265939 22676 ts_tablet_manager.cc:1398] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20240501 08:01:02.267392 17243 tablet_copy_service.cc:342] P 3d06b1f1890443da898fde2eb82daad3: Request end of tablet copy session 5aaf071b45ab469cb23321d37fad37f8-0d27d6c2cc194773a648909806f4416b received from {username='slave'} at 127.0.99.136:44473
I20240501 08:01:02.267483 17243 tablet_copy_service.cc:433] P 3d06b1f1890443da898fde2eb82daad3: ending tablet copy session 5aaf071b45ab469cb23321d37fad37f8-0d27d6c2cc194773a648909806f4416b on tablet 0d27d6c2cc194773a648909806f4416b with peer 5aaf071b45ab469cb23321d37fad37f8
I20240501 08:01:02.284420 22691 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.284829 22691 tablet_bootstrap.cc:492] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap complete.
I20240501 08:01:02.285059 22691 ts_tablet_manager.cc:1367] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.046s user 0.030s sys 0.000s
I20240501 08:01:02.285589 22691 raft_consensus.cc:348] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:02.285751 22691 raft_consensus.cc:729] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: LEARNER
I20240501 08:01:02.285833 22520 ts_tablet_manager.cc:902] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Initiating tablet copy from peer 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813)
I20240501 08:01:02.285861 22691 consensus_queue.cc:260] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: NON_VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:02.286260 22520 tablet_copy_client.cc:308] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:46813
I20240501 08:01:02.286967 17243 tablet_copy_service.cc:140] P 3d06b1f1890443da898fde2eb82daad3: Received BeginTabletCopySession request for tablet 0d27d6c2cc194773a648909806f4416b from peer d5a55f6821d84c968a42f4a5c9ad07a7 ({username='slave'} at 127.0.99.132:33311)
I20240501 08:01:02.287163 17243 tablet_copy_service.cc:161] P 3d06b1f1890443da898fde2eb82daad3: Beginning new tablet copy session on tablet 0d27d6c2cc194773a648909806f4416b from peer d5a55f6821d84c968a42f4a5c9ad07a7 at {username='slave'} at 127.0.99.132:33311: session id = d5a55f6821d84c968a42f4a5c9ad07a7-0d27d6c2cc194773a648909806f4416b
I20240501 08:01:02.287420 22691 ts_tablet_manager.cc:1398] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 08:01:02.288616 17243 tablet_copy_source_session.cc:205] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.288884 16715 tablet_copy_service.cc:342] P 64196487691e4623b690b2869aad57f7: Request end of tablet copy session d5a55f6821d84c968a42f4a5c9ad07a7-d381bdb28e1f4374b81ab23f6882fe65 received from {username='slave'} at 127.0.99.132:58205
I20240501 08:01:02.288970 16715 tablet_copy_service.cc:433] P 64196487691e4623b690b2869aad57f7: ending tablet copy session d5a55f6821d84c968a42f4a5c9ad07a7-d381bdb28e1f4374b81ab23f6882fe65 on tablet d381bdb28e1f4374b81ab23f6882fe65 with peer d5a55f6821d84c968a42f4a5c9ad07a7
I20240501 08:01:02.289546 22520 ts_tablet_manager.cc:1875] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 08:01:02.291977 22520 ts_tablet_manager.cc:1888] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 08:01:02.292178 22520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d27d6c2cc194773a648909806f4416b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.292884 22572 ts_tablet_manager.cc:902] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Initiating tablet copy from peer 64196487691e4623b690b2869aad57f7 (127.0.99.130:45219)
I20240501 08:01:02.293396 22572 tablet_copy_client.cc:308] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:45219
I20240501 08:01:02.293917 16715 tablet_copy_service.cc:140] P 64196487691e4623b690b2869aad57f7: Received BeginTabletCopySession request for tablet 5b060d9fcf294f55b5ccdd3edf5b2886 from peer e282199dc2fa4977b83ff07b86cb0690 ({username='slave'} at 127.0.99.133:51451)
I20240501 08:01:02.294051 16715 tablet_copy_service.cc:161] P 64196487691e4623b690b2869aad57f7: Beginning new tablet copy session on tablet 5b060d9fcf294f55b5ccdd3edf5b2886 from peer e282199dc2fa4977b83ff07b86cb0690 at {username='slave'} at 127.0.99.133:51451: session id = e282199dc2fa4977b83ff07b86cb0690-5b060d9fcf294f55b5ccdd3edf5b2886
I20240501 08:01:02.294397 22520 tablet_copy_client.cc:783] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.294550 22520 tablet_copy_client.cc:647] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.295054 16715 tablet_copy_source_session.cc:205] T 5b060d9fcf294f55b5ccdd3edf5b2886 P 64196487691e4623b690b2869aad57f7: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 08:01:02.296380 22572 ts_tablet_manager.cc:1875] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 08:01:02.299594 22572 ts_tablet_manager.cc:1888] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 08:01:02.299803 22572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b060d9fcf294f55b5ccdd3edf5b2886. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 08:01:02.301074 22572 tablet_copy_client.cc:783] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: tablet copy: Starting download of 0 data blocks...
I20240501 08:01:02.301234 22572 tablet_copy_client.cc:647] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: tablet copy: Starting download of 1 WAL segments...
I20240501 08:01:02.301429 22520 tablet_copy_client.cc:523] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.302520 22572 tablet_copy_client.cc:523] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 08:01:02.304055 22572 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap starting.
I20240501 08:01:02.304425 22520 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap starting.
I20240501 08:01:02.333477 22520 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.333937 22520 tablet_bootstrap.cc:492] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Bootstrap complete.
I20240501 08:01:02.334158 22520 ts_tablet_manager.cc:1367] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent bootstrapping tablet: real 0.030s user 0.028s sys 0.000s
I20240501 08:01:02.334764 22520 raft_consensus.cc:348] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:02.334956 22572 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 08:01:02.334988 22520 raft_consensus.cc:729] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d5a55f6821d84c968a42f4a5c9ad07a7, State: Initialized, Role: LEARNER
I20240501 08:01:02.335103 22520 consensus_queue.cc:260] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: NON_VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: true } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:02.335256 22572 tablet_bootstrap.cc:492] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Bootstrap complete.
I20240501 08:01:02.335443 22572 ts_tablet_manager.cc:1367] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Time spent bootstrapping tablet: real 0.031s user 0.026s sys 0.004s
I20240501 08:01:02.335515 22520 ts_tablet_manager.cc:1398] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 08:01:02.336191 17243 tablet_copy_service.cc:342] P 3d06b1f1890443da898fde2eb82daad3: Request end of tablet copy session d5a55f6821d84c968a42f4a5c9ad07a7-0d27d6c2cc194773a648909806f4416b received from {username='slave'} at 127.0.99.132:33311
I20240501 08:01:02.336278 17243 tablet_copy_service.cc:433] P 3d06b1f1890443da898fde2eb82daad3: ending tablet copy session d5a55f6821d84c968a42f4a5c9ad07a7-0d27d6c2cc194773a648909806f4416b on tablet 0d27d6c2cc194773a648909806f4416b with peer d5a55f6821d84c968a42f4a5c9ad07a7
I20240501 08:01:02.336447 22572 raft_consensus.cc:348] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } }
I20240501 08:01:02.336583 22572 raft_consensus.cc:729] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e282199dc2fa4977b83ff07b86cb0690, State: Initialized, Role: LEARNER
I20240501 08:01:02.336691 22572 consensus_queue.cc:260] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } }
I20240501 08:01:02.336988 22572 ts_tablet_manager.cc:1398] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 08:01:02.337723 16715 tablet_copy_service.cc:342] P 64196487691e4623b690b2869aad57f7: Request end of tablet copy session e282199dc2fa4977b83ff07b86cb0690-5b060d9fcf294f55b5ccdd3edf5b2886 received from {username='slave'} at 127.0.99.133:51451
I20240501 08:01:02.337816 16715 tablet_copy_service.cc:433] P 64196487691e4623b690b2869aad57f7: ending tablet copy session e282199dc2fa4977b83ff07b86cb0690-5b060d9fcf294f55b5ccdd3edf5b2886 on tablet 5b060d9fcf294f55b5ccdd3edf5b2886 with peer e282199dc2fa4977b83ff07b86cb0690
I20240501 08:01:02.355976 16957 raft_consensus.cc:1203] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.14->[2.15-2.15] Dedup: 2.15->[]
I20240501 08:01:02.356401 16563 raft_consensus.cc:1203] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 08:01:02.420472 16957 raft_consensus.cc:1203] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 08:01:02.428771 19097 raft_consensus.cc:1203] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:01:02.435524 17091 raft_consensus.cc:1203] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.11->[1.12-1.12] Dedup: 1.12->[]
I20240501 08:01:02.439853 18969 raft_consensus.cc:1203] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.12->[1.13-1.13] Dedup: 1.13->[]
I20240501 08:01:02.498402 16826 raft_consensus.cc:1203] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.12->[1.13-1.13] Dedup: 1.13->[]
I20240501 08:01:02.588007 16825 raft_consensus.cc:1203] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.11->[1.12-1.12] Dedup: 1.12->[]
I20240501 08:01:02.598737 18969 raft_consensus.cc:1203] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.11->[1.12-1.12] Dedup: 1.12->[]
I20240501 08:01:02.599869 16563 raft_consensus.cc:1203] T 5e0b477de25646eda4f9f0e4921ec118 P eca6f08d57d1453784f726199474a671 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 08:01:02.604674 17091 raft_consensus.cc:1203] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.12->[1.13-1.13] Dedup: 1.13->[]
I20240501 08:01:02.611050 16957 raft_consensus.cc:1203] T 8bc2d601fe484082b947235237dbd42f P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 08:01:02.627904 18969 raft_consensus.cc:1203] T c6841a66987744f3bc06a4b2bdbc6df2 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 08:01:02.641858 18839 raft_consensus.cc:1203] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.8->[1.9-1.9] Dedup: 1.9->[]
I20240501 08:01:02.709481 16957 raft_consensus.cc:1203] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.11->[1.12-1.12] Dedup: 1.12->[]
I20240501 08:01:02.711580 17091 raft_consensus.cc:1203] T 5b060d9fcf294f55b5ccdd3edf5b2886 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 08:01:02.763433 22227 raft_consensus.cc:1050] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7: attempting to promote NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 to VOTER
I20240501 08:01:02.764097 22227 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 2.15, Last appended by leader: 12, Current term: 2, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } }
I20240501 08:01:02.766417 18969 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 15. Preceding OpId from leader: term: 2 index: 16. (index mismatch)
I20240501 08:01:02.766849 18839 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 15. Preceding OpId from leader: term: 2 index: 16. (index mismatch)
I20240501 08:01:02.767117 19102 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 15. Preceding OpId from leader: term: 2 index: 16. (index mismatch)
I20240501 08:01:02.767200 16957 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 LEARNER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 15. Preceding OpId from leader: term: 2 index: 16. (index mismatch)
I20240501 08:01:02.768039 17091 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 15. Preceding OpId from leader: term: 2 index: 16. (index mismatch)
I20240501 08:01:02.768540 17353 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 15. Preceding OpId from leader: term: 2 index: 16. (index mismatch)
I20240501 08:01:02.768610 22227 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.769035 22910 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.769307 22911 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.769542 22243 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.772308 17222 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 15. Preceding OpId from leader: term: 2 index: 16. (index mismatch)
I20240501 08:01:02.772867 22243 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.773272 22243 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.773671 22909 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.776440 22911 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 2 LEADER]: Committing config change with OpId 2.16: config changed from index 15 to 16, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.776810 17353 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Committing config change with OpId 2.16: config changed from index 15 to 16, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.779064 17091 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Committing config change with OpId 2.16: config changed from index 15 to 16, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.780763 17222 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 2 FOLLOWER]: Committing config change with OpId 2.16: config changed from index 15 to 16, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.781484 16957 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 FOLLOWER]: Committing config change with OpId 2.16: config changed from index 15 to 16, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.782753 19102 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Committing config change with OpId 2.16: config changed from index 15 to 16, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.786360 18969 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Committing config change with OpId 2.16: config changed from index 15 to 16, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.786684 16417 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 15 to 16, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:01:02.789309 18839 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 2 FOLLOWER]: Committing config change with OpId 2.16: config changed from index 15 to 16, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.801867 16695 consensus_queue.cc:237] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 2.16, Last appended by leader: 12, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } }
I20240501 08:01:02.803805 19102 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 16. Preceding OpId from leader: term: 2 index: 17. (index mismatch)
I20240501 08:01:02.803826 18969 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 16. Preceding OpId from leader: term: 2 index: 17. (index mismatch)
I20240501 08:01:02.803856 16957 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 16. Preceding OpId from leader: term: 2 index: 17. (index mismatch)
I20240501 08:01:02.804291 18839 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 16. Preceding OpId from leader: term: 2 index: 17. (index mismatch)
I20240501 08:01:02.804340 22245 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:02.804828 22245 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:02.804873 17091 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 16. Preceding OpId from leader: term: 2 index: 17. (index mismatch)
I20240501 08:01:02.805075 22909 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:02.805510 22908 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:02.806404 17221 raft_consensus.cc:1261] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 2 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 16. Preceding OpId from leader: term: 2 index: 17. (index mismatch)
I20240501 08:01:02.808111 22245 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:02.808300 22908 consensus_queue.cc:1035] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:02.809087 22909 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 64196487691e4623b690b2869aad57f7 [term 2 LEADER]: Committing config change with OpId 2.17: config changed from index 16 to 17, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.809725 18969 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 [term 2 FOLLOWER]: Committing config change with OpId 2.17: config changed from index 16 to 17, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.812120 16406 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6fd34b65ff134bf485af45d1eccd62f9 with cas_config_opid_index 16: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:01:02.812011 16957 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 2 FOLLOWER]: Committing config change with OpId 2.17: config changed from index 16 to 17, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.812819 18839 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5aaf071b45ab469cb23321d37fad37f8 [term 2 FOLLOWER]: Committing config change with OpId 2.17: config changed from index 16 to 17, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.812880 16416 catalog_manager.cc:5484] T 6fd34b65ff134bf485af45d1eccd62f9 P 873218458f2346348be2120e543990e2 reported cstate change: config changed from index 16 to 17, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New cstate: current_term: 2 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.813925 19102 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 5f8132e5a8014a6f913ee9ac3338600a [term 2 FOLLOWER]: Committing config change with OpId 2.17: config changed from index 16 to 17, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.814522 17091 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P e282199dc2fa4977b83ff07b86cb0690 [term 2 FOLLOWER]: Committing config change with OpId 2.17: config changed from index 16 to 17, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.817241 22258 raft_consensus.cc:1050] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690: attempting to promote NON_VOTER eca6f08d57d1453784f726199474a671 to VOTER
I20240501 08:01:02.817843 22258 consensus_queue.cc:237] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } }
I20240501 08:01:02.820439 17353 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.820925 17335 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd34b65ff134bf485af45d1eccd62f9 with delete_type TABLET_DATA_TOMBSTONED (TS af70005d0c6c48bc9919cae6346f221b not found in new config with opid_index 17) from {username='slave'} at 127.0.0.1:56682
I20240501 08:01:02.820876 22947 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.821501 22950 tablet_replica.cc:330] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b: stopping tablet replica
I20240501 08:01:02.821604 17221 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.821715 22950 raft_consensus.cc:2229] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 08:01:02.822083 22950 raft_consensus.cc:2258] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 08:01:02.823427 16693 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.823705 22319 raft_consensus.cc:1050] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671: attempting to promote NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 to VOTER
I20240501 08:01:02.823751 19102 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.824349 16825 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.824613 22319 consensus_queue.cc:237] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } }
I20240501 08:01:02.825949 22118 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.826138 18969 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.826723 22118 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.827561 22947 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.827723 22260 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.828109 22260 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.828933 17220 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.829309 19097 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.829639 18838 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.829675 16693 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.830157 22435 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.830205 17222 raft_consensus.cc:2932] T 6fd34b65ff134bf485af45d1eccd62f9 P 3d06b1f1890443da898fde2eb82daad3 [term 2 FOLLOWER]: Committing config change with OpId 2.17: config changed from index 16 to 17, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.831485 16563 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 LEARNER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.832659 16959 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.833532 22333 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.834039 22333 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.834335 22333 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.834625 16957 raft_consensus.cc:1203] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.8->[1.9-1.9] Dedup: 1.9->[]
I20240501 08:01:02.835624 17091 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.836002 22949 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.836163 17353 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 08:01:02.836616 22435 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.836715 22948 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, eca6f08d57d1453784f726199474a671 (127.0.99.129) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.837412 17221 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, eca6f08d57d1453784f726199474a671 (127.0.99.129) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.837759 19102 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, eca6f08d57d1453784f726199474a671 (127.0.99.129) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.840569 22435 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.841027 22435 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 08:01:02.841958 16416 catalog_manager.cc:5484] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 11 to 12, eca6f08d57d1453784f726199474a671 (127.0.99.129) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.841989 17354 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, eca6f08d57d1453784f726199474a671 (127.0.99.129) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.842999 18970 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, eca6f08d57d1453784f726199474a671 (127.0.99.129) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.843569 16826 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, eca6f08d57d1453784f726199474a671 (127.0.99.129) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.847488 22950 ts_tablet_manager.cc:1875] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:01:02.850704 22319 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.851246 22950 ts_tablet_manager.cc:1888] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.16
I20240501 08:01:02.851330 22950 log.cc:1196] T 6fd34b65ff134bf485af45d1eccd62f9 P af70005d0c6c48bc9919cae6346f221b: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/6fd34b65ff134bf485af45d1eccd62f9
I20240501 08:01:02.851881 16407 catalog_manager.cc:4830] TS af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065): tablet 6fd34b65ff134bf485af45d1eccd62f9 (table rebalance_test_table_4 [id=0951aa8ec317451983387a5bc8546bfc]) successfully deleted
I20240501 08:01:02.852203 19102 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.853878 16563 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, eca6f08d57d1453784f726199474a671 (127.0.99.129) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.855288 16414 catalog_manager.cc:5484] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a reported cstate change: config changed from index 11 to 12, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.856186 16957 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.857575 16693 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.857993 17221 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.860095 16693 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, eca6f08d57d1453784f726199474a671 (127.0.99.129) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { replace: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.860986 17353 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.855899 18839 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.864483 17091 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, d5a55f6821d84c968a42f4a5c9ad07a7 (127.0.99.132) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.870051 16563 consensus_queue.cc:237] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } }
I20240501 08:01:02.871773 17091 consensus_queue.cc:237] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } }
I20240501 08:01:02.872246 16957 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.872284 17089 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.872670 22333 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.872817 18839 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.873005 17221 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.873457 19102 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.873513 22435 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.873834 22435 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.874274 19102 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.874574 22435 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.874635 22947 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.874881 22435 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.875121 16562 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.875622 22946 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.876461 18970 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.877007 17353 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.877092 17222 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.877213 22258 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.877576 22258 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.877725 22118 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.878471 16693 raft_consensus.cc:1261] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer e282199dc2fa4977b83ff07b86cb0690: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.879122 22949 consensus_queue.cc:1035] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.879788 22333 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.880352 18839 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.881817 16693 raft_consensus.cc:1261] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer eca6f08d57d1453784f726199474a671: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.881989 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5ae979c5adf44f3e8590ef06c71bf904 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:01:02.882444 22957 consensus_queue.cc:1035] T 5ae979c5adf44f3e8590ef06c71bf904 P eca6f08d57d1453784f726199474a671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.882596 19097 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.882887 16957 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.884105 22947 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.884559 18970 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.886060 16408 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 452b5685e6ca4793a6628d700599bdbb with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:01:02.886303 22382 raft_consensus.cc:1050] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:01:02.886935 22382 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } }
I20240501 08:01:02.888612 16413 catalog_manager.cc:5484] T 5ae979c5adf44f3e8590ef06c71bf904 P 5aaf071b45ab469cb23321d37fad37f8 reported cstate change: config changed from index 12 to 13, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "eca6f08d57d1453784f726199474a671" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.888898 16563 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.890486 17222 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.891147 17091 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.891327 16421 catalog_manager.cc:5484] T 452b5685e6ca4793a6628d700599bdbb P 873218458f2346348be2120e543990e2 reported cstate change: config changed from index 12 to 13, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "e282199dc2fa4977b83ff07b86cb0690" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.891453 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.892040 17354 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.892657 16695 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.893481 18970 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.893563 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.893925 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.894760 16826 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEARNER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.895660 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.895980 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.896390 16807 tablet_service.cc:1514] Processing DeleteTablet for tablet 452b5685e6ca4793a6628d700599bdbb with delete_type TABLET_DATA_TOMBSTONED (TS 1160c6e72055456992b5d8bf5f3b45c0 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:39058
I20240501 08:01:02.896648 19102 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.897326 22196 raft_consensus.cc:1050] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: attempting to promote NON_VOTER e282199dc2fa4977b83ff07b86cb0690 to VOTER
I20240501 08:01:02.897989 22196 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } }
I20240501 08:01:02.898797 17221 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.899647 16563 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.900164 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.901293 19097 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.901291 16957 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.901767 17220 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.901763 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.902406 22381 raft_consensus.cc:1013] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: attempt to promote peer 1160c6e72055456992b5d8bf5f3b45c0: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20240501 08:01:02.902998 18969 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.903041 16826 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEARNER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.903189 16563 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.903509 18838 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.903563 21790 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.903959 21790 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.904116 22368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.904443 22368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.904704 22368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.904946 22368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.905505 22390 raft_consensus.cc:1050] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0: attempting to promote NON_VOTER 5f8132e5a8014a6f913ee9ac3338600a to VOTER
I20240501 08:01:02.905784 16693 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.906097 22390 consensus_queue.cc:237] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:01:02.906422 22982 tablet_replica.cc:330] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0: stopping tablet replica
I20240501 08:01:02.906659 22982 raft_consensus.cc:2229] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:01:02.906965 22982 raft_consensus.cc:2258] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:01:02.907809 16692 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.908270 18839 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.908749 22382 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.908941 21790 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.909026 22982 ts_tablet_manager.cc:1875] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:01:02.909420 19102 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a [term 1 LEARNER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:01:02.910219 16563 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:01:02.910436 18968 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:01:02.911248 22755 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:02.911659 22755 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:02.911998 22755 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:02.912175 17090 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:02.913623 17354 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:01:02.914129 22755 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:02.914283 22368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.916023 17223 raft_consensus.cc:2932] T 5ae979c5adf44f3e8590ef06c71bf904 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER af70005d0c6c48bc9919cae6346f221b (127.0.99.135) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: false } } }
I20240501 08:01:02.916481 17091 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.916693 17221 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:01:02.916878 22382 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.917070 22100 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:02.918566 17335 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ae979c5adf44f3e8590ef06c71bf904 with delete_type TABLET_DATA_TOMBSTONED (TS af70005d0c6c48bc9919cae6346f221b not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:56682
I20240501 08:01:02.918923 22950 tablet_replica.cc:330] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b: stopping tablet replica
I20240501 08:01:02.919075 22950 raft_consensus.cc:2229] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:01:02.919433 22950 raft_consensus.cc:2258] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:01:02.919569 17089 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:01:02.919986 16694 raft_consensus.cc:2932] T 452b5685e6ca4793a6628d700599bdbb P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { promote: false } } }
I20240501 08:01:02.922490 22755 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:02.924391 22950 ts_tablet_manager.cc:1875] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:01:02.926522 16692 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 08:01:02.926944 22390 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 08:01:02.927091 22950 ts_tablet_manager.cc:1888] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 08:01:02.927171 22950 log.cc:1196] T 5ae979c5adf44f3e8590ef06c71bf904 P af70005d0c6c48bc9919cae6346f221b: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/5ae979c5adf44f3e8590ef06c71bf904
I20240501 08:01:02.927342 16692 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.928773 22987 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.929911 16407 catalog_manager.cc:4830] TS af70005d0c6c48bc9919cae6346f221b (127.0.99.135:39065): tablet 5ae979c5adf44f3e8590ef06c71bf904 (table rebalance_test_table_3 [id=f875840544e1404c80042fd6a2e747df]) successfully deleted
I20240501 08:01:02.930276 17222 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.930981 18838 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.931087 16562 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.931478 22982 ts_tablet_manager.cc:1888] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 08:01:02.931555 22982 log.cc:1196] T 452b5685e6ca4793a6628d700599bdbb P 1160c6e72055456992b5d8bf5f3b45c0: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/452b5685e6ca4793a6628d700599bdbb
I20240501 08:01:02.932144 16405 catalog_manager.cc:4830] TS 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131:43779): tablet 452b5685e6ca4793a6628d700599bdbb (table rebalance_test_table_0 [id=265dacd91eaf400a9f1cfc7d08a094c8]) successfully deleted
I20240501 08:01:02.934007 16957 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.934306 16416 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 reported cstate change: config changed from index 12 to 13, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.934556 22382 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:02.934985 19097 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:02.937155 18970 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:02.937738 16825 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEARNER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:02.937994 17222 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:02.938910 17353 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:02.939479 16562 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:02.940412 16415 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a reported cstate change: config changed from index 13 to 14, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:02.940760 22381 raft_consensus.cc:1050] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: attempting to promote NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 to VOTER
I20240501 08:01:02.941258 22381 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } }
I20240501 08:01:02.944169 16693 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:02.945106 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.945271 18838 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:02.945443 16563 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:02.945941 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:02.946233 16825 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEARNER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:02.946693 18970 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:02.947212 17353 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:02.947182 16694 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:02.947453 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.947583 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:02.947799 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:02.948060 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:02.948182 19097 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:02.949219 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:02.949265 18838 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:02.950551 17222 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:02.953788 22973 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:02.955444 21790 raft_consensus.cc:1050] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b: attempting to promote NON_VOTER 1160c6e72055456992b5d8bf5f3b45c0 to VOTER
I20240501 08:01:02.955436 22293 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.956393 16826 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEARNER]: Committing config change with OpId 1.13: config changed from index 12 to 13, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.957814 17353 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.959180 16693 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.959779 16563 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.960443 19097 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.961429 16418 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: config changed from index 14 to 15, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:01:02.963335 21790 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } }
I20240501 08:01:02.964211 18970 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.965276 18839 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.965670 22368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.966797 18839 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.967532 18970 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.967797 18969 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.968039 22368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:02.968667 16825 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEARNER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.970532 22368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.970386 16826 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.972862 16957 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 1.15, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } }
I20240501 08:01:02.973295 16959 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.973503 17220 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.973769 22368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.973959 22196 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.974330 16693 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.974824 22196 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.975224 16563 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:02.975759 22368 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.975984 18970 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 08:01:02.976615 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.978178 17352 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 08:01:02.978190 16825 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 08:01:02.979195 16563 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 08:01:02.979195 22382 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.979332 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.979655 22972 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.981052 19097 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 08:01:02.981801 17087 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:02.982158 18838 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 08:01:02.982373 17089 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 08:01:02.982301 22987 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.983050 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.983381 22293 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.984141 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 08:01:02.985141 16695 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.985281 22381 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.985726 17091 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } }
I20240501 08:01:02.986436 17220 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.987483 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:01:02.987476 17091 raft_consensus.cc:858] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } }.
Pending config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } }:
New config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } }
W20240501 08:01:02.987561 17091 raft_consensus.cc:1565] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Could not prepare op 'term: 1 index: 16' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } }.
Pending config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } }:
New config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } }
I20240501 08:01:02.988144 16693 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 08:01:02.989151 16957 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.989277 16413 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 reported cstate change: config changed from index 13 to 14, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.989643 16563 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.989774 16562 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.991510 22382 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 08:01:02.992385 18970 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.994073 22755 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:02.995227 16825 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.995157 17352 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:02.996448 18970 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:02.996788 17087 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:02.998556 16563 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:02.999136 16695 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:02.999974 18969 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:03.001492 16417 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 reported cstate change: config changed from index 15 to 16, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.002585 19097 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.002759 18838 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.004931 17201 tablet_service.cc:1514] Processing DeleteTablet for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with delete_type TABLET_DATA_TOMBSTONED (TS 3d06b1f1890443da898fde2eb82daad3 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:58426
I20240501 08:01:03.006847 17088 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 08:01:03.008173 17352 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:03.009685 19102 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:03.009896 17223 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:03.010331 16825 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:03.012029 22990 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:03.013037 17088 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:03.013876 16693 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.014773 17088 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:02.987825 17091 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.021785 23024 tablet_replica.cc:330] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3: stopping tablet replica
I20240501 08:01:03.022034 23024 raft_consensus.cc:2229] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:01:03.022182 18837 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, 1160c6e72055456992b5d8bf5f3b45c0 (127.0.99.131) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:03.022533 23024 pending_rounds.cc:62] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3: Trying to abort 1 pending ops.
I20240501 08:01:03.022616 23024 pending_rounds.cc:68] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3: Aborting op as it isn't in flight: id { term: 1 index: 15 } timestamp: 7022798696205869056 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "d5ecdf2c2cdb4487b0e6315f63e1b1b8" old_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: NON_VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: true } } } new_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } } }
I20240501 08:01:03.022850 23024 raft_consensus.cc:2894] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Aborting config change with OpId 1.15: Aborted: Op aborted
I20240501 08:01:03.022922 23024 raft_consensus.cc:2866] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240501 08:01:03.022965 23024 raft_consensus.cc:2258] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:01:03.023955 22978 raft_consensus.cc:1050] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7: attempting to promote NON_VOTER e282199dc2fa4977b83ff07b86cb0690 to VOTER
I20240501 08:01:03.024557 22978 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 1.16, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } }
I20240501 08:01:03.025117 23024 ts_tablet_manager.cc:1875] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:01:03.026648 17352 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 08:01:03.027029 19102 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 08:01:03.027057 22382 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:03.027431 16693 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 08:01:03.027444 22382 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:03.028821 22382 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:03.029425 18969 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 08:01:03.029547 16825 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 08:01:03.029807 22382 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:03.030093 22975 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:03.030659 18837 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 08:01:03.031193 16563 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 08:01:03.031440 22977 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:03.031610 22975 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:03.031888 22975 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.17: config changed from index 16 to 17, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.031245 17091 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: NON_VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: true } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.033809 16421 catalog_manager.cc:5484] T ecd1ff8b89a84669bd8001ea731be7dc P 873218458f2346348be2120e543990e2 reported cstate change: config changed from index 8 to 9, 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:03.034876 17088 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 LEARNER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240501 08:01:03.035200 17353 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.037655 22943 raft_consensus.cc:1050] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7: attempting to promote NON_VOTER 873218458f2346348be2120e543990e2 to VOTER
I20240501 08:01:03.038324 22943 consensus_queue.cc:237] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:03.039690 22382 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240501 08:01:03.041160 16825 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.042438 16563 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.043231 23024 ts_tablet_manager.cc:1888] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.15
I20240501 08:01:03.043305 23024 log.cc:1196] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 3d06b1f1890443da898fde2eb82daad3: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/d5ecdf2c2cdb4487b0e6315f63e1b1b8
I20240501 08:01:03.044885 17091 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.045315 17088 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:03.045295 16419 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 16 to 17, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.045678 16407 catalog_manager.cc:4830] TS 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813): tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 (table rebalance_test_table_2 [id=b24c2218c98e4ee296163018ea520a92]) successfully deleted
I20240501 08:01:03.046120 22909 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:03.048130 19102 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:03.048408 18837 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:03.048542 16563 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:03.050690 22245 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:03.051105 22245 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:03.051192 16957 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:03.051192 17353 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:03.052292 22908 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:03.052541 22909 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:03.052651 22908 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:03.052721 17223 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:03.052534 18970 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.055415 17352 consensus_queue.cc:237] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } }
I20240501 08:01:03.055662 22909 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:03.056360 22227 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.056800 18837 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.057348 19102 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.059813 16695 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.060304 18837 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, e282199dc2fa4977b83ff07b86cb0690 (127.0.99.133) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.061117 16563 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.063187 18969 raft_consensus.cc:1261] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2 [term 1 LEARNER]: Refusing update from remote peer 64196487691e4623b690b2869aad57f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 08:01:03.063726 22911 consensus_queue.cc:1035] T d381bdb28e1f4374b81ab23f6882fe65 P 64196487691e4623b690b2869aad57f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 08:01:03.065817 18969 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.067952 16419 catalog_manager.cc:5484] T d381bdb28e1f4374b81ab23f6882fe65 P eca6f08d57d1453784f726199474a671 reported cstate change: config changed from index 12 to 13, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "64196487691e4623b690b2869aad57f7" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.068970 19102 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.070645 17088 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.071480 16959 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 1.17, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } }
I20240501 08:01:03.072686 18837 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:03.073127 22988 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:03.073833 16825 consensus_queue.cc:237] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } }
I20240501 08:01:03.074182 18970 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:03.075302 22988 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:03.076117 16563 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.076586 23001 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.076737 22251 raft_consensus.cc:1050] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3: attempting to promote NON_VOTER 5aaf071b45ab469cb23321d37fad37f8 to VOTER
I20240501 08:01:03.077353 22251 consensus_queue.cc:237] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } }
I20240501 08:01:03.077755 16695 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:03.078120 22990 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:03.078178 17354 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.079195 23001 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.080070 16562 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:03.080255 18970 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.080943 16826 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.080883 23071 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.081333 22841 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.081431 16563 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.081877 17354 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.082441 21973 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.082726 21973 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.082881 19097 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.083454 16692 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.083492 16958 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:03.083724 17091 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.083909 22841 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.084223 23002 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.084373 16958 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEARNER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.084473 23001 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.085992 22841 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.086339 22990 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:03.086453 17223 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.086586 17091 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.086654 22990 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:03.087749 18837 raft_consensus.cc:1261] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8 [term 1 LEARNER]: Refusing update from remote peer 3d06b1f1890443da898fde2eb82daad3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.088297 17223 raft_consensus.cc:2932] T d381bdb28e1f4374b81ab23f6882fe65 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 873218458f2346348be2120e543990e2 (127.0.99.137) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false replace: true } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.088907 22755 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:03.089663 23000 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.090010 21973 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.090327 21973 consensus_queue.cc:1035] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.090987 16827 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240501 08:01:03.091354 17355 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240501 08:01:03.091961 22977 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240501 08:01:03.092674 18970 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240501 08:01:03.092710 17223 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:03.093623 17090 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240501 08:01:03.094522 22976 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240501 08:01:03.094859 22976 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240501 08:01:03.094802 17090 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:03.095549 19097 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:03.096160 22990 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [term 1 LEADER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:03.096794 16405 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ecd1ff8b89a84669bd8001ea731be7dc with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:01:03.098131 22977 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240501 08:01:03.098939 16415 catalog_manager.cc:5484] T ecd1ff8b89a84669bd8001ea731be7dc P 5f8132e5a8014a6f913ee9ac3338600a reported cstate change: config changed from index 9 to 10, VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) evicted. New cstate: current_term: 1 leader_uuid: "1160c6e72055456992b5d8bf5f3b45c0" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:03.098909 16561 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:03.100359 16825 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:03.101161 16695 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:03.101452 17088 raft_consensus.cc:1261] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer af70005d0c6c48bc9919cae6346f221b: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 08:01:03.101895 19097 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240501 08:01:03.102077 18839 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240501 08:01:03.102072 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fa0d919844a349549b1ad6ffbe440837 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:01:03.102442 22976 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240501 08:01:03.102416 16562 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:03.102841 22976 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240501 08:01:03.103195 16694 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240501 08:01:03.104460 16693 raft_consensus.cc:1261] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer 1160c6e72055456992b5d8bf5f3b45c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 08:01:03.104871 23002 consensus_queue.cc:1035] T ecd1ff8b89a84669bd8001ea731be7dc P 1160c6e72055456992b5d8bf5f3b45c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 08:01:03.105824 22990 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:03.107141 22991 consensus_queue.cc:1035] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 08:01:03.107270 16693 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:03.107388 18839 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:03.107555 16956 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:03.110308 22977 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240501 08:01:03.110183 16418 catalog_manager.cc:5484] T fa0d919844a349549b1ad6ffbe440837 P af70005d0c6c48bc9919cae6346f221b reported cstate change: config changed from index 14 to 15, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "af70005d0c6c48bc9919cae6346f221b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 08:01:03.110626 18968 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:03.112542 17088 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:03.114866 16827 raft_consensus.cc:2932] T fa0d919844a349549b1ad6ffbe440837 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, VOTER 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } attrs { replace: true } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } }
I20240501 08:01:03.116969 22977 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.120932 16415 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 reported cstate change: config changed from index 17 to 18, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 08:01:03.121393 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:01:03.122473 18837 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.123278 17351 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.123483 17090 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.125203 16541 tablet_service.cc:1514] Processing DeleteTablet for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with delete_type TABLET_DATA_TOMBSTONED (TS eca6f08d57d1453784f726199474a671 not found in new config with opid_index 18) from {username='slave'} at 127.0.0.1:50112
I20240501 08:01:03.125244 17201 tablet_service.cc:1514] Processing DeleteTablet for tablet fa0d919844a349549b1ad6ffbe440837 with delete_type TABLET_DATA_TOMBSTONED (TS 3d06b1f1890443da898fde2eb82daad3 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:58426
I20240501 08:01:03.126714 16693 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.128132 18970 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.130453 19102 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5f8132e5a8014a6f913ee9ac3338600a [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.135140 23024 tablet_replica.cc:330] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3: stopping tablet replica
I20240501 08:01:03.135334 23024 raft_consensus.cc:2229] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:01:03.135704 23024 raft_consensus.cc:2258] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:01:03.136848 16956 consensus_queue.cc:237] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 1.18, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } }
I20240501 08:01:03.137132 23024 ts_tablet_manager.cc:1875] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:01:03.136966 16827 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER eca6f08d57d1453784f726199474a671 (127.0.99.129) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false replace: true } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.140432 17088 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20240501 08:01:03.140673 16827 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20240501 08:01:03.140949 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240501 08:01:03.143049 23086 tablet_replica.cc:330] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671: stopping tablet replica
I20240501 08:01:03.143205 23086 raft_consensus.cc:2229] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 08:01:03.143625 23086 raft_consensus.cc:2258] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 08:01:03.143783 23024 ts_tablet_manager.cc:1888] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.14
I20240501 08:01:03.143937 23024 log.cc:1196] T fa0d919844a349549b1ad6ffbe440837 P 3d06b1f1890443da898fde2eb82daad3: Deleting WAL directory at /tmp/dist-test-taskrYyCEf/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714550345291789-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/fa0d919844a349549b1ad6ffbe440837
I20240501 08:01:03.144009 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240501 08:01:03.147374 23086 ts_tablet_manager.cc:1875] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P eca6f08d57d1453784f726199474a671: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 08:01:03.148437 16407 catalog_manager.cc:4830] TS 3d06b1f1890443da898fde2eb82daad3 (127.0.99.134:46813): tablet fa0d919844a349549b1ad6ffbe440837 (table rebalance_test_table_5 [id=4793f084d4c04a6399628961193fb55e]) successfully deleted
I20240501 08:01:03.148378 23071 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 3d06b1f1890443da898fde2eb82daad3 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.149756 17352 raft_consensus.cc:2932] T ecd1ff8b89a84669bd8001ea731be7dc P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 873218458f2346348be2120e543990e2 (127.0.99.137) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "5f8132e5a8014a6f913ee9ac3338600a" member_type: VOTER last_known_addr { host: "127.0.99.138" port: 37529 } attrs { promote: false } } }
I20240501 08:01:03.151890 18839 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20240501 08:01:03.153391 17354 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20240501 08:01:03.153349 18839 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.153815 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240501 08:01:03.154592 22381 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240501 08:01:03.156653 16562 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P eca6f08d57d1453784f726199474a671 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.157016 22975 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [term 1 LEADER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.157634 17354 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P af70005d0c6c48bc9919cae6346f221b [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.160245 16407 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d5ecdf2c2cdb4487b0e6315f63e1b1b8 with cas_config_opid_index 18: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 08:01:03.160284 16418 catalog_manager.cc:5484] T 0d27d6c2cc194773a648909806f4416b P 5aaf071b45ab469cb23321d37fad37f8 reported cstate change: config changed from index 9 to 10, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d06b1f1890443da898fde2eb82daad3" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.160955 16693 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 64196487691e4623b690b2869aad57f7 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20240501 08:01:03.160801 16826 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.162537 18970 raft_consensus.cc:1261] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 873218458f2346348be2120e543990e2 [term 1 FOLLOWER]: Refusing update from remote peer d5a55f6821d84c968a42f4a5c9ad07a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20240501 08:01:03.162508 22978 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240501 08:01:03.163000 16827 raft_consensus.cc:2932] T 0d27d6c2cc194773a648909806f4416b P 1160c6e72055456992b5d8bf5f3b45c0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 5aaf071b45ab469cb23321d37fad37f8 (127.0.99.136) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "eca6f08d57d1453784f726199474a671" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43741 } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } } peers { permanent_uuid: "3d06b1f1890443da898fde2eb82daad3" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 46813 } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false replace: true } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: NON_VOTER last_known_addr { host: "127.0.99.132" port: 41987 } attrs { promote: true } } }
I20240501 08:01:03.163172 17088 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.165370 18839 raft_consensus.cc:2932] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P 5aaf071b45ab469cb23321d37fad37f8 [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.166741 22955 raft_consensus.cc:1050] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671: attempting to promote NON_VOTER d5a55f6821d84c968a42f4a5c9ad07a7 to VOTER
I20240501 08:01:03.167255 23022 consensus_queue.cc:1035] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P d5a55f6821d84c968a42f4a5c9ad07a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240501 08:01:03.167218 16418 catalog_manager.cc:5484] T d5ecdf2c2cdb4487b0e6315f63e1b1b8 P e282199dc2fa4977b83ff07b86cb0690 reported cstate change: config changed from index 18 to 19, VOTER 5f8132e5a8014a6f913ee9ac3338600a (127.0.99.138) evicted. New cstate: current_term: 1 leader_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "64196487691e4623b690b2869aad57f7" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 45219 } } peers { permanent_uuid: "d5a55f6821d84c968a42f4a5c9ad07a7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 41987 } } peers { permanent_uuid: "af70005d0c6c48bc9919cae6346f221b" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 39065 } } peers { permanent_uuid: "5aaf071b45ab469cb23321d37fad37f8" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 34371 } attrs { promote: false } } peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 41463 } attrs { promote: false } } peers { permanent_uuid: "873218458f2346348be2120e543990e2" member_type: VOTER last_known_addr { host: "127.0.99.137" port: 33839 } attrs { promote: false } } peers { permanent_uuid: "1160c6e72055456992b5d8bf5f3b45c0" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43779 } attrs { promote: false } } }
I20240501 08:01:03.167351 22955 consensus_queue.cc:237] T 8bc2d601fe484082b947235237dbd42f P eca6f08d57d1453784f726199474a671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e282199dc2fa4977b83ff07b86cb0690" member_type: VOTER last_known_addr { host: "127.0.99.133"